Slides https://docs.google.com/presentation/d/1kc2hm5i7Dnt5C1xBEFnfURKgW3XjR32uvPe0e6LZ0SU In this 30 minute session you will learn about which challenges we face testing the Drupal 8 Open Social distribution and how we are (still) solving them. Intended audience & takeaways This talk is interesting for developers with a intermediate experience level. In this session we will show you: What challenges we face testing the Open Social distribution Demonstration of our test automation tool (which is not just testing the distribution) An overview of the Open Social testing tools landscape Data on how it improved our quality Next steps & learnings we still need to tackle Background Drupal distributions can contain a lot of features for specialized use cases. And even though most distributions have unit tests and functional tests which help cover most of the use cases, it might not be enough. People often use distributions and customize it to their liking. This then brings a lot of variety in cases you’d like to see covered in the distribution. At Open Social we provide such a full-featured distribution and we encountered the following issues: Inability to test all possible scenarios Declining confidence in delivered quality Bugs discovered after a new release Lack of flexibility in added features Accommodating external contributors Mid 2018 we created an automated tool to test all the changes in the distribution, before they land in the release, also for each of our customer projects. This unique scenario gave us the opportunity to gain valuable insights in how the distribution changes act in “real life” scenarios. At the end of the session the audience will leave the room with an understanding of the challenges that come with testing a Drupal distribution. By showing real life examples of how thinking outside of the box can increase quality and boost confidence, we hope to inspire all attendees and give them the tools they need to make their own projects a success.