feat(terragrunt): add support for gitlab-tags datasource

[lstoeferle]

Changes

Hi everyone,
Unfortunately the PRs for Terragrunt dependency updates do not include any changelog, since the terragrunt manager currently only supports git-tags, github-tags and terraform-module datasources, which are not able to fetch changelogs from out GitLab Releases.

Context

• Terragrunt Manager not yet able to fetch GitLab release notes #27885

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

• Code inspection only, or
• Newly added/modified unit tests, or
• No unit tests but ran on a real repository, or
• Both unit tests + ran on a real repository

[viceice]

Please reduce test fixtures, we don't need them all. just one for each new datasource is enough.

[viceice]

why do you change so many fixtures? I think you mostly only need to change a few samples to trigger the new code paths

[lstoeferle]

Just re-arranged them a bit to have a better grouping like having all GitTag and skipReason fixtures together

[viceice]

Just re-arranged them a bit to have a better grouping like having all GitTag and skipReason fixtures together

please do that in a separate PR

[lstoeferle]

Alright, I've reverted the extra refactorings.

[viceice]

next time you should do the test url renaming in a separate PR 🤗

[renovate-release]

🎉 This PR is included in version 37.253.0 🎉

The release is available on:

• GitHub release
• npm package (@latest dist-tag)
• 37.253.0

Your semantic-release bot 📦🚀

[1337andre]

with this PR, all of our gitlab MergeRequest was closed by renovate.

renovate.json

{
...
  "regexManagers": [
    {
      "description": "Update terraform modules pinned to a digest.",
      "fileMatch": [
        ".*hcl$"
      ],
      "matchStrings": [
        "\\s*#\\s*renovate:(\\s{1,}datasource=(?<dataSource>.*))(?:\\s{1,}versioning=(?<versioning>[a-z]+))(?:\\s{1,}registryUrl=(?<registryUrl>.*))\\s*source\\s*=.*\\..*\/(?<packageName>.*?)(\\?|\\.git.*\\?)ref=(?<currentValue>.*)\\\""
      ],
      "datasourceTemplate": "{{#if dataSource}}{{{dataSource}}}{{else}}github-releases{{/if}}",
      "depNameTemplate": "{{{replace '.*\/' '' depName}}}",
      "versioningTemplate": "{{#if versioning}}{{{versioning}}}{{else}}semver{{/if}}"
    },
...
}

terragrunt.hcl

terraform {
  # renovate: datasource=gitlab-tags versioning=semver registryUrl=https://git.mycompany.de
  source = "git::https://git.mycompany.de/aws/aws-eks/modules/aws-eks-cluster.git?ref=2.3.1"

  # For local live development
  # source = "./aws-eks-cluster"
}

logs

DEBUG: Datasource 404 (repository=hycloud/aws/aws-eks/aws-eks-account, baseBranch=prod-k8s-dev)
       "datasource": "gitlab-tags",
       "packageName": "https://git.mycompany.de/hycloud/aws/aws-eks/modules/aws-eks-cluster.git",
       "url": "https://gitlab.com/api/v4/projects/https%3A%2F%2Fgit.mycompany.de%2Fhycloud%2Faws%2Faws-eks%2Fmodules%2Faws-eks-cluster.git/repository/tags?per_page=100"
DEBUG: Failed to look up gitlab-tags package https://git.mycompany.de/hycloud/aws/aws-eks/modules/aws-eks-cluster.git (repository=hycloud/aws/aws-eks/aws-eks-account, baseBranch=prod-k8s-dev, packageFile=cluster/terragrunt.hcl, dependency=https://git.mycompany.de/hycloud/aws/aws-eks/modules/aws-eks-cluster.git)

[lstoeferle]

Seems like the API call url is wrongly constructed now, it should be sth. like:

- "url": "https://gitlab.com/api/v4/projects/https%3A%2F%2Fgit.mycompany.de%2Fhycloud%2Faws%2Faws-eks%2Fmodules%2Faws-eks-cluster.git/repository/tags?per_page=100"
+ "url": "https://gitlab.com/api/v4/projects/hycloud%2Faws%2Faws-eks%2Fmodules%2Faws-eks-cluster/repository/tags?per_page=100"

[viceice]

@rarkins revert?

[lstoeferle]

I maybe found the issue. It seems like the GitTags datasource expecpts the full URL in the packageName field (see here), where the GitLab datasource expects only the project path in the packageName field (see here).

Currently the manager always sets the whole URL as packageName (see here). So most probably there needs to be some logic based on the detected datasource.