Slides: http://www.slideshare.net/DataStax/datastax-highly-available-spark-stream-processing-with-confluent-platform-and-datastax-enterprise-wei-deng-cassandra-summit-2016-66482850 | In this presentation, we will discuss the essential components in building a full pipeline of real-time stream processing on IoT data (Smart Meter data as the example) that is highly available, highly scalable and highly performant. The focus will be on best practice in achieving high availability of all components involved (kafka-rest service for producer, kafka schema registry, kafka brokers, zookeeper, DSE Spark masters, DSE Spark workers, DSE Spark executors, DSE Spark streaming application/driver), but we will also touch on scalability and performance considerations for such a pipeline. We will discuss various failure scenarios and how to compensate for them to avoid downtimes. About the Speakers Ryan Svihla Advanced Response Engineer, DataStax Wei Deng Solutions Architect, DataStax