Go to content

Machine Learning with Scala on Spark by Jose Quesada

This video was recorded at Scala Days Berlin 2016 follow us on Twitter @ScalaDays or visit our website for more information http://scaladays.org Abstract: What new superpowers does it give me? The machine learning libraries in Apache Spark are an impressive piece of software engineering, and are maturing rapidly. What advantages does Spark.ml offer over the older technologies that inspired its design? At Data Science Retreat we've taken a real-world dataset and worked through the stages of building a predictive model -- exploration, data cleaning, feature engineering, and model fitting -- in several different frameworks. We'll show what it's like to work with Spark.ml, and compare it to other widely used frameworks (in R and python) along several dimensions: ease of use, productivity, feature set, and performance. In some ways Spark.ml is still rather immature, but it also conveys new superpowers to those who know how to use it. We hope to inspire you to join us in using and improving it.

June 13, 2016