Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate flex release #337

Open
westes opened this issue Mar 29, 2018 · 15 comments · May be fixed by #480
Open

Automate flex release #337

westes opened this issue Mar 29, 2018 · 15 comments · May be fixed by #480
Milestone

Comments

@westes
Copy link
Owner

westes commented Mar 29, 2018

Currently, flex releases are manual. This takes time and is error prone.

We would like for release tags to trigger flex releases.

Such a system needs to:

  • build the flex release archives.
  • generate release notes based on the NEWS file.
  • notify the flex-announce mailing list (NB it is ok if that email needs approval).
  • increment the recorded current and previous version numbers of flex based on the release number from the release tag and what flex's version numbering is recorded as.

When we have that, it will be possible to release flex whenever an important fix or feature change appears on master or when sufficient time has passed since the previous release.

@westes westes added this to the 2.6.5 milestone Mar 29, 2018
@westes westes mentioned this issue Dec 25, 2018
@GitMensch
Copy link
Contributor

Once I get time to finish #337, this will happen.

Originally posted by @westes in #412 (comment)

This is the single open issue in the 2.6.5 milestone and 18 items were already closed in the 2.6.6 milestone [although I'm not sure if they were "just closed" without any actual commit, in this case those should be adjusted to have no milestone attached].

As there obviously is no "release often" happening - can we please have the 2.6.5 release (now containing changes of nearly 4 years) done "at least this time" manually?

@Mightyjo
Copy link
Contributor

Mightyjo commented Feb 9, 2021

I'm getting the hang of scripting automated release builds as Travis CI "deployments." So far I know what to do with a new tag. (Still figuring out how to get the flex version number in there, but making progress.)

When you say, "build the flex release archives," are you wanting to re-build historical dist tarballs?

@GitMensch
Copy link
Contributor

When you say, "build the flex release archives," are you wanting to re-build historical dist tarballs?

No, that is about running make dist -> creation of the distribution tarballs.

I'm getting the hang of scripting automated release builds as Travis CI "deployments."

Does this mean you may be able to help @westes with a PR including a script and notes how to setup that as CI that at least:

  • take the take
  • running make distcheck
  • if it passes making a GH release out of the tag and attach the generated tarballs and attaching the first NEWS entry to it as release note
  • optional, as it could be done later and manually in the meantime: "increment the recorded current and previous version numbers of flex" (whatever that means, I guess it would be a separate CI step that is triggered at the end of the first and would commit a change to the repo [or create a PR to do this] with som numbering)

?

@Mightyjo
Copy link
Contributor

Yep, that's the gist of it. I've worked out how to automate just about everything between the github api, Travis, automake, and some utilities. I'll probably add a maintainer script and a make target to kick it off.That's all for new releases, though.

Getting arbitrary old build systems into Travis is possible but tricky so I'd be glad not to do that.

@Mightyjo
Copy link
Contributor

Mightyjo commented Feb 15, 2021

I have Travis making a release from a tag now.

Working out how to make the gpg signatures. Are those made according to a specific formula? I'm guessing they are are detached signatures of the file hashes.

Also thinking about the NEWS file. Will building the update from the milestone be sufficient? I have a plan for how to do that with octokit.

@Croydon
Copy link

Croydon commented Feb 15, 2021

You probably should not use Travis unless westes is willing to a) regularly ask Travis for more CI credits or b) pay for it.

Travis is not freely available in general anymore https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing

Additionally, even before the pricing change, there seems to be a misconfiguration of Travis in this repository. Travis did not run anymore since May 2018.

I would recommend looking into GitHub Actions instead.

@Mightyjo
Copy link
Contributor

Super. Thanks for the heads up. I got the builds working across their whole matrix without them every mentioning this has changed.

@Mightyjo
Copy link
Contributor

@westes what do you think about switching to GitHub Actions from Travis CI? Looks like it's a little more hassel to do multi-arch builds, but I think coordinating releases will be simpler.

@westes
Copy link
Owner Author

westes commented Feb 18, 2021

It makes sense to switch to github actions. For now, I'd say ignore the mswindows and osx builds since they consume minutes at higher rates.

@Mightyjo
Copy link
Contributor

Rgr that. I'll get the fundamentals set up on linux/amd64 to start.

@Mightyjo
Copy link
Contributor

Mightyjo commented Mar 5, 2021

Okay, got the first chunk in place. @westes, requesting feedback on the NEWS file format and workflow. The NEWS file I'm generating is pretty basic. It's just issue numbers, titles, and labels (when available). That's all inserted at the top of the file, though, so it's possible to add more friendly news nearby just before triggering the automated release workflow.

The workflow is triggered when you close a milestone. The tag and release names are taken from the milestone title.
Start condition: A milestone exists with a name like "v2.6.5"
Entry: The milestone is closed.
Steps:

  1. The new version number is derived from the milestone title.
  2. Closed issues in the milestone are collected, formatted, and inserted into the NEWS file.
  3. The version number is updated in Flex sources (TODO)
  4. A new commit containing the NEWS and updated sources is created and pushed by the GitHub Actions runner.
  5. The new commit is tagged with the version number by the GitHub Actions runner.
    If the above were successful
  6. The new tag is checked out, built, and distchecked
  7. The dist tarballs are signed (TODO)
  8. The GitHub source tarballs are signed (TODO; perhaps redundant)
  9. GitHub Actions creates a release based on the milestone name
  10. The dist artifacts are uploaded to the new release
    End condition: A new tag refers to the master branch and a release built from that tag includes dist tarballs.

The gh-actions branch of my fork is in the end condition right now. The workflow files are in .github/workflows/milestone* if you want to take a look. I have a daily build workflow and some tests in there, too.

NB: The milestone workflow only runs on the default branch. I switch mine to gh-actions when I'm testing it.

@Mightyjo Mightyjo linked a pull request Mar 27, 2021 that will close this issue
@Mightyjo
Copy link
Contributor

PR is ready for comment. The 2.6.5-TEST "release" on my fork shows how it all turns out. I can kick off new demos without much trouble - as designed :)

@GitMensch
Copy link
Contributor

@westes You've requested the automated adjustments via github-actions and there's a PR for that. Would you please review the PR? If the "automated" part does not fit for whatever reason we could "downgrade" the build to not take care of the releases and only run build + make distcheck, still providing automated tests and release tarballs for people to use from the CI.

@GitMensch
Copy link
Contributor

Friendly ping @westes after a year.

@gruenich
Copy link

gruenich commented Feb 17, 2024

I don't see what you want to automate any further.

  1. build the flex release archives.
    => This is done by GitHub if you create a new release. This is easy.
  2. generate release notes based on the NEWS file.
    => Auto-generated notes do not work. The lists are cluttered with uninteresting noise. From my experience, if you get (or you create one) a pull request for a change that should be part of the release notes, force people to add the information as part of the pull request. This requires some discipline during development, not for a release.
  3. notify the flex-announce mailing list (NB it is ok if that email needs approval).
    => Is this really that much work to write a brief message including a copy of the GitHub release page?
  4. increment the recorded current and previous version numbers of flex based on the release number from the release tag and what flex's version numbering is recorded as.
    => How should an automated mechanism decide whether you intend to do a major, minor, or patch-level increment? Again, I don't think this is the real trouble for a release.

The main burden I see is 2, creating the release notes. Changing your merge policy might help you for incoming requests. I offer to help to update the release notes since the last release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants