Jenkins is an ‘open source automation server’ commonly used for Continuous Integration and Continuous Delivery of software projects by many tech companies. It was first released in 2005 when it was originally known as Hudson. Its large collection of available plugins (~1800!), particularly Pipeline, enables teams to automate common operations tasks, particularly building, testing, and releasing.
All of that being said: it’s time to consider alternatives, especially for new software projects. This article outlines the limitations and pain points of the current incumbent CI/CD system and then suggests alternatives.
Plugin Complexity
Regarding the plugin ecosystem: some plugins depend on others in order to function- and can depend on specific versions of those dependencies and Jenkins itself. This can lead to difficult upgrade paths due to ‘dependency hell’ and the reluctance to update plugins for fear of jobs breaking. Automatic updating of Jenkins plugins is therefore nigh impossible. This requires building a suite of tools to build and deploy test environments to verify combinations of up-to-date Jenkins and plugins, and the manual task of coordinating and triggering the upgrade itself with minimal impact on engineering productivity.
Does your R&D department have the expertise and time to manage your Jenkins installation and perform updates?
Operational Complexity
Also, Jenkins requires care and feeding. Examples of common operational responsibilities are tuning JVM parameters to set the proper heap size and garbage collection behavior, as well as capacity planning and right-sizing resources for Jenkins workers. Additionally, since pipelines are often defined using Groovy scripts, those also require monitoring and maintenance to ensure that they are working properly and are performant. Finally, while a pipeline can be represented in a JenkinsFile and checked into revision control, not all plugins are supported- meaning that there are many situations where pipelines are configured by hand.
All of these manual tasks add up to an operational overhead similar to an entire production system. Sadly, many teams depend on unmaintained Jenkins instances due to higher priorities.
Does your R&D department have sufficient staffing and budget to operate your Jenkins infrastructure?
Security Posture
Jenkins installations (as are any CI/CD systems) are common targets of threat actors, as they can be exploited for supply chain attacks and source code exfiltration.
Once again, while there is indeed an extensive collection of plugins, Jenkins installations that use a large number of them create a similarly large attack surface. Each plugin’s maturity and the maintainer’s level of response to known security holes differ from each other, and the Jenkins project itself created a formal process to manage this problem.
Some vulnerabilities are not fixed, or worse- some plugins are no longer maintained at all. Meaning: that if your pipeline depends on a vulnerable plugin, you may get stuck with the choice of removing it and breaking your pipeline- or simply accepting the risk.
An even worse outcome is the Jenkins installation not getting attention at all from a security standpoint- meaning the likelihood of a breach is even higher.
What is the threat model of your Jenkins infrastructure? What is your current level of risk, and the worst-case scenario if made manifest?
Alternatives
If Jenkins is already in use by a team or project, the principle “if it ain’t broke, don’t fix it” certainly applies. However, for new projects, evaluate your options before choosing Jenkins by default.
Offerings from Git Hosting Providers
Vendors such as Github, Gitlab, and Bitbucket all offer the capability to build and run CI/CD pipelines that are typically written in YAML and committed to your project’s repository. They are tightly integrated with pull requests and merges. They will of course provide a more limited feature set than what Jenkins offers, however, if it meets all (or most!) of your project’s needs, starting there allows you to outsource the responsibility of running the build/test/deploy infrastructure.
SaaS Offerings
Similarly, there are other vendors outside of the Git hosting ecosystem that offer CI/CD:
- TeamCity: formerly known as TravisCI. Was considered the defacto CI system for GitHub prior to their introduction of Github Actions.
- CircleCI: has become popular in recent years, however experienced a high-profile security incident earlier this year that compromised customer credentials. Their incident report is available here.
- HarnessCI: offers many features as part of its platform, including a CI/CD offering based on Drone. See below.
- Bitrise specializes exclusively in mobile applications.
There are many, many more options beyond the ones above. However, depending on a SaaS vendor for build/test/deploy may not meet team requirements. Let’s look at additional choices that allow for self-hosting.
Self-Hosted Options
- Drone, provided by HarnessCI as open-source, is an interesting option. Similar to the above, the pipeline’s definition is YAML-based and stored in the code repo. In addition to local and SSH execution, each step of the pipeline supports being executed in its own ephemeral Docker container, meaning you can completely customize each step’s runtime according to its needs independently from others. It was built for deployment on Kubernetes, however could easily be run on VMs or cloud instances. There is also a plugin ecosystem, and the capability to express pipelines in the Starlark language (based on Python).
- KrakenCI also supports containerization and Starlark, and focuses on supporting large-scale of workers.
- Argo is a CD mechanism based around GitOps for managing Kubernetes workloads. It has been adopted by a growing list of users, many of whom are recognizable brands.
If There Are No Alternatives
If your team must use Jenkins:
- Build pipelines out of a minimum level of plugins and ensure they are actively maintained and updated when security flaws are discovered. The cloud vendor Cloudbees offers Beekeeper, which can help select high-quality plugins and keep them up to date.
- Use Jenkinsfiles to keep pipelines in revision control.
- Keep Groovy scripts simple and leave complexity to build scripts.
- Monitor your Jenkins infrastructure to be informed of performance issues or outages.
- Backup your Jenkins data periodically.
- Finally consider using Cloudbees’ CI platform- keep Jenkins without the operational burden!
Conclusion
Jenkins has provided automation for building, testing, and releasing of software for almost two decades, however has shown its age due to pain points in its plugin system, its overall security, and the effort required to operate it. While Jenkins may remain the best choice for your current projects, consider trying different CI/CD solutions before adopting it for new ones.
I’m currently helping my clients better ship their software with higher quality. Reach out if your organization needs similar guidance!