Update sha256 values in regex managers

[whilp]

What would you like Renovate to be able to do?

We have configured a regex manager to update file assets published in GitHub releases. We download these releases using a custom Bazel rule that we wrote. For this reason, and because these are files (more similar to Bazel's http_file than http_archive), we can't apply the (awesome) support for updating http_archive defs.

With the following configuration, renovate knows how to update the version string in the URL:

        {
            "fileMatch": "^WORKSPACE$",
            "matchStrings": [
                "# renovate (?<datasource>.*)\n.*\"https://github.com/(?<depName>.*?)/releases/download/(?<currentValue>.*?)/.*\""
            ]
        }

For stuff in our WORKSPACE like:

# ibazel
bintool(
    name = "ibazel",
    urls = {
        "darwin": (
            # renovate github-releases
            "https://github.com/bazelbuild/bazel-watcher/releases/download/v0.11.1/ibazel_darwin_amd64",
            "3ab365182dcb3bfa9e6ac5dacefc7ee29f876f60dc656653306f1ae38a942b2a",
        ),
        "linux": (
            # renovate github-releases
            "https://github.com/bazelbuild/bazel-watcher/releases/download/v0.11.1/ibazel_linux_amd64",
            "c01fa95b53b5ecde8341b424027abb5f3e74190e9c14a44e2aa87791d1d8b6c1",
        ),
    },
)

Here, the urls attribute is a string-list dict, where our bintool rule expects the values to be two-item tuples like (url, sha256).

Describe the solution you'd like

We would like renovate to update the sha256 values in addition to the version numbers. To do this, perhaps it would make sense to generalize the existing sha256 support in the bazel manager, such that we can extend our regex to include the sha and the updater will know how to update it.

Describe alternatives you've considered

http_archive (and therefore the bazel manager) can't handle single file assets like these executables. And regex managers can only handle the version update.

Additional context

Thank you!

[rarkins]

I need to think more about how this could be done. It's hard to make it generic because what you need is pretty specific to github releases.

[whilp]

Thanks for the quick response! And I agree -- this would be a valuable (if perhaps tricky) feature.

Given that we successfully figure out the new URL, perhaps getHashFromURL could work without much modification:

renovate/lib/manager/bazel/update.ts

Lines 47 to 67 in b74a491

	async function getHashFromUrl(url: string): Promise<string | null> {
	const cacheNamespace = 'url-sha256';
	const cachedResult = await globalCache.get<string | null>(
	cacheNamespace,
	url
	);
	/* istanbul ignore next line */
	if (cachedResult) {
	return cachedResult;
	}
	try {
	const hash = await fromStream(http.stream(url), {
	algorithm: 'sha256',
	});
	const cacheMinutes = 3 * 24 * 60; // 3 days
	await globalCache.set(cacheNamespace, url, hash, cacheMinutes);
	return hash;
	} catch (err) /* istanbul ignore next */ {
	return null;
	}
	}

Maybe it'd be useful to think of GitHub releases (in concert with a regex) as a full-fledge manager?

[strayer]

I like to do something similar in Dockerfiles when installing tools that don't provide GPG signed releases:

FROM debian:bullseye

RUN apt-get update && apt-get install -y curl libdigest-sha-perl xz-utils && rm -rf /var/lib/apt/lists

ARG SHELLCHECK_VERSION=0.7.1
ARG HADOLINT_VERSION=1.17.5
ARG SHELLCHECK_CHECKSUM="64f17152d96d7ec261ad3086ed42d18232fcb65148b44571b564d688269d36c8"
ARG HADOLINT_CHECKSUM="20dd38bc0602040f19268adc14c3d1aae11af27b463af43f3122076baf827a35"

RUN set -xe; \
  curl -L -o shellcheck.tar.xz "https://github.com/koalaman/shellcheck/releases/download/v${SHELLCHECK_VERSION}/shellcheck-v${SHELLCHECK_VERSION}.linux.x86_64.tar.xz"; \
  curl -L -o hadolint "https://github.com/hadolint/hadolint/releases/download/v${HADOLINT_VERSION}/hadolint-Linux-x86_64"; \
  { \
  echo "$SHELLCHECK_CHECKSUM  shellcheck.tar.xz"; \
  echo "$HADOLINT_CHECKSUM  hadolint"; \
  } > checksum.txt; \
  shasum -c checksum.txt; \
  tar xf shellcheck.tar.xz; \
  mv -t /usr/local/bin/ shellcheck-v*/shellcheck hadolint; \
  chmod +x /usr/local/bin/shellcheck /usr/local/bin/hadolint

Would be great if Renovate could update the checksums too when opening PRs for those versions. (I do realize this example isn't so easy because based on the ARGs it is not clear which exact file would need to be hashed, but the general idea still applies)