Jenkins World 2017

Having migrated 100+ FreeStyle CI jobs to declarative CD style pipelines we learned a lot. We hit a few bumps, had to challenge & change our development process and write some code, all whilst having a bit of fun along the way. We will discuss this acquired knowledge to help you be more productive.

Last year our Company’s internal Jenkins instance had become overwhelmed as we had grown. It contained jobs from all our teams and departments, everyone was an administrator, there was no change control practice and plugin upgrades affected everyone. This was a result of the startup nature at CloudBees, and the tipping point had come when we wanted to do even more testing of our products which lead to long queue times for simple jobs . So the decision was made to move off this single instance to a new cluster, where each team could have its own master and greater control over its build environment with Docker. But rather than migrating what we had we took the opportunity to change and improve our teams’ setup and dogfood many of the shiny new features that were in development at the time.Taking the approximately 150 jobs and replacing them with declarative pipelines configured automatically from GitHub using custom marker files (a proprietary feature) we now no longer have to create jobs for new plugins as this is performed automatically, we even have a way for an individual plugin to replace the default pipeline all controlled via the SCM. However things were not plain sailing, dogfooding caused us to find many issues, our existing processes were not suitable for automation and we often broke the pipeline due to lack of testability/testing. Due to these issues we often had to write some code and work out how to test the pipeline to prevent us breaking all our jobs, as well as devise a workable strategy that would enable us to reach our end goal of continuous delivery. This presentation will share our experiences and techniques for dealing with a similar migration.