Add goal that works with Maven CI Friendly Versions and set property (revision) based on git tag

[jimisola]

Maven supports Maven CI Friendly Versions since version 3.5.

It allows the user to set a placeholder

<version>${revision}</version>

that can be replaced from command line using:

e.g.

mvn -Drevision=1.0.0-SNAPSHOT clean package
Our use-case is that you want this revision property (can be configurable) to be set by a Maven plugin using tag information from the scm (typically, git).

The revision property should be set as follows:

note: will be set to variants of nr of commits

1. if no tags in repo: set to 0.0.1- (e.g. 0.0.1-23) (left-padded due to Maven bug comparing qualifer as a string)
2. if tags exist
   1. if latest commit is tagged (assume a release): set revision to tag (e.g. 1.0.1)
   2. if it is a previous tag: use the tag, bump patch level, set build number to and add <-SNAPSHOT> (e.g. 0.5.1 -> 0.5.2--SNAPSHOT

That is the basics. I was contemplating writing a plugin for it, but we'll rather so it incorporated in into this plugin.

Build number will be handled correctly by Maven as can be seen here:

java -jar ${MAVEN_HOME}/lib/maven-artifact-3.9.6.jar 1.1 1.2.1-1 1.2.1-1-SNAPSHOT 1.2.1-9 1.3
Display parameters as parsed by Maven (in canonical form and as a list of tokens) and comparison result:
1. 1.1 -> 1.1; tokens: [1, 1]
   1.1 < 1.2.1-1
2. 1.2.1-1 -> 1.2.1-1; tokens: [1, 2, 1, [1]]
   1.2.1-1 > 1.2.1-1-SNAPSHOT
3. 1.2.1-1-SNAPSHOT -> 1.2.1-1-snapshot; tokens: [1, 2, 1, [1, [snapshot]]]
   1.2.1-1-SNAPSHOT < 1.2.1-9
4. 1.2.1-9 -> 1.2.1-9; tokens: [1, 2, 1, [9]]
   1.2.1-9 < 1.3
5. 1.3 -> 1.3; tokens: [1, 3]

[lfvjimisola]

Would you approve a PR with this functionality?

[olamy]

maybe you should use an extension for this rather than a plugin.

https://maven.apache.org/examples/maven-3-lifecycle-extensions.html

[lfvjimisola]

Ended up creating our own plugin. Could use Maven SCM API and it did feel right to add the JGit dependency.

We haven't published it yet, so we could potentially provide it as an extension but probably need some guidance then.

Also, we are in need of this now really and I'm not sure how long it would take for a PR and a release to happen.

[slawekjaranowski]

Ended up creating our own plugin. Could use Maven SCM API and it did feel right to add the JGit dependency.

Your code with JGit looks simple, Maven SCM Api add support for others tools - I will try with it.
But if it will be complicated we can start with JGit 😄

We haven't published it yet, so we could potentially provide it as an extension but probably need some guidance then.

As next goal will be easier than extension.

Also, we are in need of this now really and I'm not sure how long it would take for a PR and a release to happen.

Release can happen in a few days after merge.

I watch a repository and when we have new feature or bug fix I will simply release.
Now we have for next release: https://github.com/mojohaus/versions/milestone/17?closed=1
Last release was in November ... so it will be good time for next one.

[lfvjimisola]

@slawekjaranowski I tried with Maven SCM API, but it seemed like it was to simple for this use-case. Could be that I missed out on something. I reckon it might will be configurable later on how the git repo should be traversed from the starting commit (branch only etc), so for flexibility I'd prefer JGit.

You wrote "I will try with it". Was that as in you or a suggestion for me?

As next goal will be easier than extension.

Not following here either.
What do you mean with as next goal? Do you mean that creating a goal is favorable over an extension?
It seems to me that a new regular goal is a better option.

We would rather see this feature incorporated in the Versions Maven plugin as it involves versioning of POMs but with another angle, i.e. instead of changing the actual POM.xml once uses CI/CD and tags.

I'm a bit stressed for time now (3 weeks since I wrote here) and we need this feature asap.
That said, I'm willing to provide an initial PR if I can get some help to get it through 🙏
E.g. I'm not sure what would be the best way to write tests since we need a git repostory for that (would git-bundle be an option?).

As for our code we will donate it without any copyright claims or what not.

However, the plugin contains a modified (VersionInformation) of code that is copyrighted by Karl Heinz Marbaise under MIT License for mojohaus. Is there a solution to this (licenses are MIT for build-helper and Apache 2.0 for Versions)?

[slawekjaranowski]

@slawekjaranowski I tried with Maven SCM API, but it seemed like it was to simple for this use-case. Could be that I missed out on something. I reckon it might will be configurable later on how the git repo should be traversed from the starting commit (branch only etc), so for flexibility I'd prefer JGit.

You wrote "I will try with it". Was that as in you or a suggestion for me?

Suggestion for you 😄

As next goal will be easier than extension.

Not following here either. What do you mean with as next goal? Do you mean that creating a goal is favorable over an extension? It seems to me that a new regular goal is a better option.

I think about new goal.

We would rather see this feature incorporated in the Versions Maven plugin as it involves versioning of POMs but with another angle, i.e. instead of changing the actual POM.xml once uses CI/CD and tags.

I'm a bit stressed for time now (3 weeks since I wrote here) and we need this feature asap.

It is open source project maintained by volunteers in their free time, so I can not give you any promise about timelines.

That said, I'm willing to provide an initial PR if I can get some help to get it through 🙏 E.g. I'm not sure what would be the best way to write tests since we need a git repostory for that (would git-bundle be an option?).

We use maven-invoker-plugin for IT test, there are possibility to use pre - setup and post - verify script
I suggest to try preparing testing repo in pre script

As for our code we will donate it without any copyright claims or what not.

However, the plugin contains a modified (VersionInformation) of code that is copyrighted by Karl Heinz Marbaise under MIT License for mojohaus. Is there a solution to this (licenses are MIT for build-helper and Apache 2.0 for Versions)?

We can copy as is with preserving license header .... or change @khmarbaise what do you think

[lfvjimisola]

1. Ok. Suggestion understood ;)
2. A new goal it is.
3. I did not imply that we expected some else to get this feature asap. I'm simply stating that we have a plugin that will work for us for now. I've been involved off and on in open source for 20 years or so and fully understand the volunteer aspect. We all do what we can with the time that we have. I've seen how active you are in the Maven community.
4. Ok. When I have the time I'll look into maven-invoker-plugin for IT then. Thanks.

[lfvjimisola]

@slawekjaranowski Would it be possible to elaborate a little moe on how to use the maven invoker plugin for testing with git repos?

I'm looking at e.g. it-823-ranges-update-report-001 and it-set-scm-tag-004 but it's not clear to me how I use maven invoker plugin to either use a pre-set git repository or to create on of the fly. This in order to have test data.

I'm thinking that I need to use JGit to setup and then regular JUnit.

I honestly think that testing Maven plugin is a mess. Shall I use the maven-plugin-testing-harness even?

[slawekjaranowski]

With invoker, create a file: selector.groovy with content like:

try {
    // smoke test if we have a needed tools
    def gitVersion = "git --version".execute()
    gitVersion.consumeProcessOutput(System.out, System.out)
    gitVersion.waitFor()
    return gitVersion.exitValue() == 0
} catch (Exception e) {
    // some error occurs - we skip a test
    return false
}

if git will not available on system test will be skipped

next you create: setup.groovy, with content like:

void exec(String command) {
    def proc = command.execute(null, basedir)
    proc.consumeProcessOutput(System.out, System.out)
    proc.waitFor()
    assert proc.exitValue() == 0 : "command '${command}' return status: " + proc.exitValue()
}

def testFile = new File(basedir, 'test.txt')
testFile << 'content'

exec('git init')
exec('git add test.txt')
exec('git status')
exec('git commit -m first-commit')
exec('git tag first-tag')

and finally create a verify.groovy to check result

you can execute only one test by:

mvn verify -P run-its -pl versions-maven-plugin -DskipTests -Dinvoker.test=test-name

[lfvjimisola]

Thank you. I'll start work on this this week (currently, upgrading our internal pipelines).

During that upgrade, I realized that you sometimes need to bypass the version being set based on git (e.g. when testing).

In Python the tools use environment variables such as

• SETUPTOOLS_SCM_PRETEND_VERSION=0.0.1
• POETRY_DYNAMIC_VERSIONING_BYPASS=0.0.1

Are there existing environment variables used by Versions Maven Plugin and if so, how are they named?

Otherwise my suggestion is:

• VERSIONS_MAVEN_PLUGIN_ SCM_USE_VERSION=x.y.z

[lfvjimisola]

@slawekjaranowski

The maven wrapper specified in the project uses version 3.8.6 of Maven, but:

[INFO] [ERROR] Rule 3: org.apache.maven.enforcer.rules.version.RequireMavenVersion failed with message:
[INFO] [ERROR] You need at least Maven 3.9.0 to build MojoHaus projects.

It seems a the Maven Wrapper haven't been updated in sometime and I know that there have been some significant changes to it (provided a PR myself if I recall right).

I'm having issues running mvnw when behind our company proxy - both directly and via our internal JFrog Artifactory (using a locally installed Maven works fine). I think it would be good to update the Maven Wrapper in Versions to rule out and issue that has been resolved.

Off topic:
I've tried bypassing the proxy but can't get it to work (tried with export MAVEN_OPTS="-Djava.net.useSystemProxies=true" among other things) and using our internal JFrog Artifactory (using MVNW_REPOURL) but no luck. I hoped that MVNW_REPOURL would work but mvnw chops of the path and using only host.domain.se as BASE_URL instead of host.domain.se/artifactory/sysdev-maven2.

[slawekjaranowski]

• VERSIONS_MAVEN_PLUGIN_ SCM_USE_VERSION=x.y.z

What for such env will be used ... if it is a parameter for Mojo you simply should use a filed with annotation @Parameters

[slawekjaranowski]

The maven wrapper specified in the project uses version 3.8.6 of Maven, but:

eh ... don't use a wrapper it can be generate more problems like proxy etc ...

[INFO] [ERROR] Rule 3: org.apache.maven.enforcer.rules.version.RequireMavenVersion failed with message:
[INFO] [ERROR] You need at least Maven 3.9.0 to build MojoHaus projects.

What command do you execute ? 3.9.0 is required only for release as I remember.

[jimisola]

The maven wrapper specified in the project uses version 3.8.6 of Maven, but:

eh ... don't use a wrapper it can be generate more problems like proxy etc ...

[INFO] [ERROR] Rule 3: org.apache.maven.enforcer.rules.version.RequireMavenVersion failed with message:
[INFO] [ERROR] You need at least Maven 3.9.0 to build MojoHaus projects.

What command do you execute ? 3.9.0 is required only for release as I remember.

Ok. That explains it. Before I saw your command in #1071 (comment) I ran ./mvnw -B release:prepare release:perform as per Releasing since there was no examples under Contributing.

[jimisola]

• VERSIONS_MAVEN_PLUGIN_ SCM_USE_VERSION=x.y.z

What for such env will be used ... if it is a parameter for Mojo you simply should use a filed with annotation @Parameters

Normally yes. But, but there are occasions for a SCM plugin goal that needs to be compatible with CI systems when you can't set/use the Maven Plugin parameter. Without those two environment variables (used by Python build systems hatch and poetry) it would have been really difficult to write tests for our build system among other things.

The option to by pass the plugin setting the property can be be particularly useful in CI/CD pipelines or in build environments where the SCM metadata might not be fully available or accessible. Typically, during testing, you might want to temporarily override the automatic versioning behavior without modifying the configuration files or the SCM tags.

[jimisola]

I believe that the creation of a new git repository in setup.groovy must be created outside the existing git repository in order to avoid a "nestled" repository. There are tests that will require that a git repo does not exist at all.

I've been able to find any information, but is there a best practice for mojohaus plugins on how to use the invoker to create a temporary directory in setup.groovy, pass the information to verify.groovy (unless verify.groovy remains in the same cwd as setup.groovy) and then assuring that the temporary directory is cleaned up after the test?

From what I have read and verified the invoker will clone the integration test project to target/it/ which is still nested.

I could really use a pointer then as I would like to do the following:

1. selector: check for git
2. setup:
   1. create git repo for testing in a temporary directory that does not contain a parent git repositry
   2. run versions maven plugin goal set-version-from-scm-tag (specified in invoker.properties)
3. verify: assert that the version expected is set by checking the maven log