Go to content

ScalaIO - Manuel Bernhardt - A tale of futures, actors & pipes: using Akka for large data migration

Talenthouse is a Los-Angeles based start-up offering an online platform for creative collaboration and brand advertisement with over 3 million users. In the end of 2013 it became clear that the existing platform was not performing well enough anymore and that a new system had to be built, and all of its data migrated - in one "big-bang" week-end. This is the story of how Akka was used to build an entirely reactive, high-throughput, failure-tolerant data migration pipeline, fetching data from multiple databases, cloud storages, YouTube, SoundCloud and other locations.

October 23, 2014