Go to content

With BIG data comes BIG responsibility; The curious case of BOL.COM by Mark Nijenhuis

Bol.com is the largest online retailer in the Netherlands and Belgium and is still growing at a staggering rate. For gathering business intelligence (BI) insights we have a team of twelve BI engineers. The growth of data, users and complexity have forced our team of to rethink our traditional Oracle based data warehouse structure. Currently we are moving towards a more hybrid BI solution based on Microservices at a BIG data scale, using Hadoop, HBASE, Oracle, etc. In this presentation Debarshi and Mark will discuss the unique challenges we have faced and are still facing with petabytes of data along with design decisions, tooling landscape and architectural choices. We have created and explored various toolings and concepts for solving our problems. We discuss concepts pertaining to Data quality at big data scale, large scale batch scheduling, job monitoring, ETL processing, reporting, continuous deployment and team autonomy for BI at scale. Essentially, all the hacks to get our systems towards awesomeness. The presentation will guide audience through transformation of legacy (vintage) BI towards more scalable BI setup as well as mental and techincal block that came along the way

November 9, 2015