Go to content

Google Dataflow: The New Open Model for Batch and Stream Processing by Sara Robinson & Robert Kubis

In 2004 Google published the MapReduce paper, a programming model that kick-started big data as we know it. Ten years later, Google introduced Dataflow - a new paradigm, integrating batch and stream processing in one common abstraction. This time the offer was more than a paper, but also an open source Java SDK and a cloud managed service to run it. In 2016 big data players like Cask, Cloudera, Data Artisans, PayPal, Slack, Talend joined Google to propose Dataflow for incubation at the Apache Software Foundation - Dataflow is here, not only unifying batch and streaming, but also the big data world. In this talk we are going to review Dataflow's differentiating elements and why they matter. We’ll demonstrate Dataflow’s capabilities through a real-time demo with practical insights on how to manage and visualize streams of data. Sara Robinson is a Developer Advocate on Google's Cloud Platform team, where she helps with developer relations through online content, outreach and events. She has a bachelor’s degree in Business and International Studies from Brandeis University. When she's not programming, she can be found running, listening to country music, or finding the best ice cream in SF. Robert Kubis is a Developer Advocate for the Google Cloud Platform based in London specializing in Container, Storage and Scalable technologies. Before joining Google, Robert collected over 10 years of experience in Software Development and Architecture. He has driven multiple full-stack application developments at SAP with a passion for distributed systems, containers and databases. In his spare time he enjoys following tech trends, good restaurants, traveling and improving his photographing skills :) More info at https://www.linkedin.com/in/kubisrobert [CTP-1409]

November 7, 2016