Using Elixir for scheduling cloudnative DAG workflows - Lucjan Suski
This video was recorded at ElixirConf EU Virtual 2020 - https://www.elixirconf.eu/archives/virtual_x_2020/index.html More great virtual tech conferences - https://codesync.global --- Using Elixir for scheduling cloudnative DAG workflows by Lucjan Suski ABSTRACT Current state-of-art DAG workflow schedulers (AirFlow, Luigi) do not seem to leverage the power of on-demand cloud computing instances. Therefore, Lucjan and his team used Elixir's distributed capabilities to write a powerful, yet simple, scheduler for their data processing pipeline. Lucjan will talk about the various approaches to running external processes in Elixir, monitoring and displaying progress UI in LiveView and testing and the challenges. --- THE SPEAKER - Lucjan Suski Working in webdev industry since he was 12 years old. After 5 years of day-to-day Rails and React he co-founded a SaaS product company which evolved from an Elixir side project. --- ElixirConf EU Website: https://www.elixirconf.eu/ Twitter: https://twitter.com/ElixirConfEU