Continuous Delivery Details
Mockito champions true continuous delivery in the Open Source.
- Introduction
- Continuous Delivery
- Versioning
- Why 2.1.0 and not 2.0.0?
- Lessons learned
- Too many beta versions!
For motivation see Continuous Delivery Overview page.
Mockito implements continuous delivery since 2014. In early 2017 the team decided to change the release model, for motivation and the design spec see Continuous Delivery Pipeline 2.0.
Continuous Delivery allows interested parties to access to the latest versions including betas, thus allowing a drastically shortened feedback cycle. Examples of quick feedback cycles are issues like #360, #335 or #237. All these issues pointed to specific beta versions which allowed us to track down very quickly which pull request or commit was the culprit.
Mockito Continous Delivery Pipeline is implemented using Shipkit library and configured in "shipkit.gradle" file. The details are described in README.md file.
Before the introduction of the continuous delivery drone, the team intuitively followed the semver scheme. Upon the the introduction of the introduction of the continuous delivery, the version scheme was the same.
The 1.10.x
line, could follow semver x
being the build number :
1.10.<build number>
That leads to the following qestion...
tldr: the first release of Mockito 2.X is 2.1.0 and you can download this artifact from maven central.
To answer this question, we need to go back in December 2014. With this commit the version scheme didn't change, the beta version was just marked with the beta
marker (label) to indicate Mockito 2.x beta builds had started.
This lead to versions like :
2.0.<build number>-beta
Indeed the official semver definition also states that
Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
While this notion is seemingly innocent, we were now unable to publish Mockito 2.0.0
: maven and gradle with version ranges would fetch the very latest beta release instead of our official release! Moreover, maven-central indicated that, while 2.0.0
was published later, it was superseded in various patch updates. e.g.
1.10.18 < 1.10.19 < 2.0.0-beta < 2.0.0 < 2.0.1-beta < 2.0.111-beta
It took us quite some time to address that problem during the 2.x beta phase. But we did :
There were multiple workarounds to this problem (see #595).
-
The first workaround considered was unpublishing the beta releases of Mockito. However, the recent problems of such an invasive action with
left-pad
in the NPM ecosystem (read more) made clear that this was not a reasonable solution. Removing beta releases would break a significant number of projects, which is unacceptable. -
The second solution considered was to publish
2.0.0
anyways and describe the version caveat clearly in the release notes and in the upgrade guide. While this is certainly possible, the issue of maven central indicating the incorrect latest version remains. Moreover, end-users might not read the notes or guide and download the incorrect beta version thinking of it as next verion after2.0.0
. -
Lastly we considered skipping releasing Mockito
2.0.0
and instead jump to2.1.0
immediately.This is the solution Mockito team chose.
While this is certainly confusing at first, it does not break applications of end-users and fixes the issue with maven central.
The chosen solution also tweaked the version to append the build number after the beta
label. e.g.
1.10.19 < 2.0.0-beta < 2.0.1-beta < 2.0.111-beta < 2.1.0-beta.112 < 2.1.0 < 2.2.0.beta.1 < 2.2.0 < ...
-
If you are employing continuous delivery and publishing beta builds or wishes to set it up on your project, make sure to not increment the patch version. Instead, append the increment after the tag:
2.0.0-beta.15
. If you realized at this point you are using a problematic version scheme, do not remove the published artifacts. Look for a solution that is the least invasive to the end-users. This might differ from package-manager ecosystem and individual libraries/frameworks. -
When you get the version scheme and continuous delivery correct, the benefits are very valuable. The feedback cycle is greatly shortened and debugging an issue can be done a lot quicker. End-users or contributors can easily bisect an issue by switching artifacts rather easily, instead of building source and using
git bisect
. This lowers the bar for end-users resulting in more and precise feedback.
Publishing every single beta version caused confusion. Due to life priorities the active work on Mockito 2 was unfortunately intermittent. This caused prolonged beta cycle with hundreds of beta versions in Maven Central. This caused confusion and issue #595 discusses this problem at length. The issue will addressed by #618.