We all know the power of 101 – fundamental knowledge. In practice, what is fundamental leads to advanced applications and continuous pursuit of improvement. Today […]
In our platform we often have to fetch data from various locations (e.g. S3, SFTP, API) and in various formats (CSV, TSV, JSON, XML) because we have an incredibly diverse client and publisher catalog and each one provides their data in their own unique way. As we have grown over time, we’ve amassed a large list of microservices, processes, and configuration that handle these different data sources and files. The biggest issue that we’ve run into with these services is that the various portions of the data pipeline do not interact as well as we would like, so if there are any errors in that process for any reason, it can be difficult to track down where it is at times. We have begun to feel some strain from this, so we’re abstracting and centralizing as much as we can.
In the past years, we’ve seen an explosion of chat bots across multiple industries. Many times we are asked what can a chat bot do, and how would it benefit our product? In our experience, chat bots need to be tailored specifically to what a client would want otherwise, there is a very generic feeling to these bots (much like calling into an automated call center). So how can we make a bot succeed in an area crowded with thousands of existing bots?
If you’re reading this from the US, there’s a good chance that you were impacted in one way or another by the distributed denial of service attacks that took place against Dyn on October 21st. The attack resulted in disruptions for millions of users of many large and popular sites. Smaller but still important pieces of infrastructure were also impacted.
So did we see anything happen in metasearch? In short, yes. For many of our clients, it was business as usual, but a few were more impacted based on their publisher portfolio, technology stack, and global traffic mix. Here’s what we’re seeing so far.
At Koddi we’re always looking for ways to increase the speed and stability of our platform. One of our latest projects is speeding up our daily ingestion of data.
All of our data is initially stored in flat files on S3 before being loaded into our database. We’re currently in the process of integrating Apache Spark into our load process to drastically increase the speed of our loads. One problem we ran into is that S3 doesn’t behave like a normal file system in terms of read and write speeds. This is where Alluxio comes in. Alluxio is a “memory speed virtual distributed storage system” which lies between frameworks (such as Spark, MapReduce, Flink, etc.) and a storage system (Amazon S3, Google Cloud Storage, HDFS, Ceph, etc.). This allows for dramatically faster data access, with some users seeing a 30x increase in data throughput. For a more in-depth overview of Alluxio, see their documentation.
Early in the week, we noticed an unusual spike in traffic in certain geographies in the southeast United States. Upon digging in, we found that the uptick in traffic was coming primarily from States in the path of Hurricane Matthew. We found that many users were searching for hotels further inland in their respective states.
While the spike did impact properties of all types in one way or another, we found that lower ADR hotels saw a 341% increase in click volume for hotels in Florida, Georgia, South Carolina, and North Carolina, while only seeing a 14% increase in all other US states.
We’re seeing a new Deals placement for certain hotel queries on Google, now placed above the top ad slot. Clicking it brings you into the map results with the “Nearby Deal” preselected. From our observations, it appears that the Deals are nearby properties with similar ratings and amenities.
Artificial Intelligence (AI) is a term that is applied to an entity that mimics cognitive functions associated with human behavior. This mimicry occurs as the […]