A while back we wrote about how we integrated Redshift into our platform, and, more recently, we've discussed how to get Alluxio up and running in a DC/OS cluster. Now, we'd like to spend a little time to discuss how we bring these and other technologies together to create an event-driven data pipeline. The Problem In our platform we often...
One of the major challenges of building an advanced bidding and reporting platform is dealing with the large amounts of data we see come in and out of our system. Our database grows rapidly on a day to day basis. So we must ensure that we have enough storage to manage it as well as a powerful enough computing solution...