gitlab-ci include:local not working

[j0nm1]

What Renovate type are you using?

I'm using renovate/renovate:21.25.0 with Gitlab CI.

Describe the bug

Gitlab CI include:local is still not working. I tried to add support with the following pr and now Renovate recognizes the dependencies but is unable to update: #6630

Relevant debug logs

[...]
DEBUG: latest commit (repository=jonmi/gitlab-include-demo)
        "branchName": "master",
        "latestCommitDate": "2020-07-08 20:00:35 +0000"
 DEBUG: getLabels(https://index.docker.io, library/node, latest) (repository=jonmi/gitlab-include-demo)
 DEBUG: getManifestResponse(https://index.docker.io, library/node, latest) (repository=jonmi/gitlab-include-demo)
 DEBUG: Package releases lookups complete (repository=jonmi/gitlab-include-demo)
        "baseBranch": "master"
 DEBUG: packageFiles with updates (repository=jonmi/gitlab-include-demo)
        "config": {
          "gitlabci": [
            {
              "packageFile": ".gitlab-ci.yml",
              "manager": "gitlabci",
              "deps": [
                {
                  "depName": "alpine",
                  "replaceString": "alpine",
                  "autoReplaceStringTemplate": "{{depName}}{{#if newValue}}:{{newValue}}{{/if}}{{#if newDigest}}@{{newDigest}}{{/if}}",
                  "datasource": "docker",
                  "depType": "image",
                  "depIndex": 0,
                  "updates": [],
                  "warnings": [],
                  "skipReason": "unsupported-value"
                }
              ]
            }
          ],
          "gitlabci-include": [
            {
              "packageFile": ".gitlab-ci.yml",
              "manager": "gitlabci-include",
              "deps": [
                {
                  "depName": "node",
                  "currentValue": "12.18.2-alpine",
                  "replaceString": "node:12.18.2-alpine",
                  "autoReplaceStringTemplate": "{{depName}}{{#if newValue}}:{{newValue}}{{/if}}{{#if newDigest}}@{{newDigest}}{{/if}}",
                  "datasource": "docker",
                  "commitMessageTopic": "Node.js",
                  "depType": "image",
                  "depIndex": 0,
                  "updates": [
                    {
                      "fromVersion": "12.18.2-alpine",
                      "toVersion": "12.18.2",
                      "newValue": "12.18.2",
                      "newMajor": 12,
                      "newMinor": 18,
                      "updateType": "minor",
                      "isSingleVersion": true
                    },
                    {
                      "fromVersion": "12.18.2-alpine",
                      "toVersion": "14.5.0",
                      "newValue": "14.5.0",
                      "newMajor": 14,
                      "newMinor": 5,
                      "updateType": "major",
                      "isSingleVersion": true
                    }
                  ],
                  "warnings": [],
                  "sourceUrl": "https://github.com/nodejs/node",
                  "dockerRegistry": "https://index.docker.io",
                  "dockerRepository": "library/node"
                }
              ]
            }
          ]
        }
 DEBUG: branchifyUpgrades (repository=jonmi/gitlab-include-demo)
 DEBUG: 1 flattened updates found: node (repository=jonmi/gitlab-include-demo)
 DEBUG: Returning 1 branch(es) (repository=jonmi/gitlab-include-demo)
  WARN: No github.com token has been configured. Skipping release notes retrieval (repository=jonmi/gitlab-include-demo)
        "manager": "gitlabci-include",
        "depName": "node",
        "sourceUrl": "https://github.com/nodejs/node"
 DEBUG: config.repoIsOnboarded=true (repository=jonmi/gitlab-include-demo)
 DEBUG: processRepo() (repository=jonmi/gitlab-include-demo)
 DEBUG: Processing 1 branch: renovate/docker-node-12.x (repository=jonmi/gitlab-include-demo)
 DEBUG: Calculating hourly PRs remaining (repository=jonmi/gitlab-include-demo)
 DEBUG: currentHourStart=1594238400000 (repository=jonmi/gitlab-include-demo)
 DEBUG: PR hourly limit remaining: 2 (repository=jonmi/gitlab-include-demo)
 DEBUG: Enforcing prConcurrentLimit (20) (repository=jonmi/gitlab-include-demo)
 DEBUG: 0 PRs are currently open (repository=jonmi/gitlab-include-demo)
 DEBUG: PR concurrent limit remaining: 20 (repository=jonmi/gitlab-include-demo)
 DEBUG: Calculated maximum PRs remaining this run (repository=jonmi/gitlab-include-demo)
        "prsRemaining": 2
 DEBUG: processBranch with 1 upgrades (repository=jonmi/gitlab-include-demo, dependencies=node, branch=renovate/docker-node-12.x)
 DEBUG: Setting baseBranch to master (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: Setting baseBranch to master (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: latest commit (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
        "branchName": "master",
        "latestCommitDate": "2020-07-08 20:00:35 +0000"
 DEBUG: getBranchPr(renovate/docker-node-12.x) (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: branchExists=false (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: Branch has 1 upgrade(s) (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: recreateClosed is false (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: findPr(renovate/docker-node-12.x, chore(deps): update node docker tag to v12.18.2, !open) (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: prAlreadyExisted=false (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: Checking schedule(at any time, null) (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: No schedule defined (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: Branch needs creating (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: Using reuseExistingBranch: false (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
 DEBUG: manager.getUpdatedPackageFiles() (repository=jonmi/gitlab-include-demo, branch=renovate/docker-node-12.x)
        "reuseExistingBranch": false,
        "branchName": "renovate/docker-node-12.x"
  WARN: Cannot find replaceString in current file content (repository=jonmi/gitlab-include-demo, packageFile=.gitlab-ci.yml, branch=renovate/docker-node-12.x)
        "depName": "node",
        "existingContent": "include:\n  - local: 'tag.yml'\n\nstages:\n- pass\n\npass:\n  stage: pass\n  image: alpine\n  script:\n    - echo pass\n",
        "replaceString": "node:12.18.2-alpine"
 DEBUG: No content changed (repository=jonmi/gitlab-include-demo, packageFile=.gitlab-ci.yml, branch=renovate/docker-node-12.x)
        "depName": "node"
[...]

To Reproduce

Demo Gitlab Repo: https://gitlab.com/jonmi/gitlab-include-demo

Additional context

It seems like the problem is that Renovate searches in the wrong file for the version string. I'm not 100% sure what's necessary but after looking into the existing code I think custom extractAllPackageFiles and update methods need to be implemented (as done for maven). Maybe somebody has the time to look into it.

[rarkins]

Can you help me understand how this functionality works? Is the included local file typically another gitlab CI file, in which case will we end up with duplicate extracted dependencies? Or is it a file we wouldn't normally index?

[j0nm1]

The include:local keyword is used to import other yml files so that you don't have to define all jobs in one file but in separate files (e.g. for different branches & merge requests). These yml files cannot be picked up by Renovate directly because there is no regex to find them, you need to resolve the include statements from the .gitlab-ci.yml.

Official documentation: https://docs.gitlab.com/ee/ci/yaml/#includelocal

[rarkins]

OK, in that case we're going to need the extract function to add a new field to such cases, named something like editFile (so we can know the version is not in the packageFile). Is that something you can add? It won't fix it immediately, but I'll then add a feature to our auto-update so that it replaces in the editFile instead of packageFile.

[j0nm1]

Ok, so implementing a extractAllPackageFiles is the starting point? If so, I will try do to it tomorrow.

[rarkins]

No need for a new function. We just need the extra field added to the existing results

[j0nm1]

Including the editFile seems pretty easy by just adding includedDep.editFile = includeObj.local; to this part:

renovate/lib/manager/gitlabci-include/extract.ts

Lines 55 to 57 in 74b6669

	for (const includedDep of includedDeps) {
	deps.push(includedDep);
	}

But currently doAutoReplace does not respect this property as you mentioned here #6745 (comment)

[rarkins]

It needs to be resolved though, not relative

[rarkins]

Needs:

• extract function should add editFile param to applicable dependencies
• autoReplace function needs to recognize this and replace in that file instead of the package file

[fionera]

We are using renovate 23.28.0 and noticed that the remote includes are also not working. The file gets matched for the manager but the manager doesnt detect anything in the file.

[ibacalu]

using this in 23.49.7 and renovate correctly follows the include and detects the changes but fails when applying because it tries to apply them on the main .gitlab-ci.yml file instead of the individual included file paths

[l0nax]

I've created a PR (#8805) to fix this problem.

[erzz]

Related - but not the same so if you want me to open a seperate issue please say :)

I followed the PR's and they will be great for local: includes - but there are various other types of include which also result in a similar error

INFO: Cannot find replaceString in current file content. Was it already updated? (repository=my-group/my-project, packageFile=.gitlab-ci.yml, branch=renovate/docker-haynes-jacoco2cobertura-1.x)

The common types of include we use are:
template: as you know about seeing as renovate-runner project uses it
remote: could be any HTTP and or GIT repo

I don't see a way where either of those cases could be handled by updating someone else's repo of course :) But can it be handled better? Perhaps ...

• Still raise the MR (as I want to be made aware) rather than failing silently (as far as the target repo is concerned). Then at least I can choose to ignore or take it up with the author of the included file
• Raise an issue?
• Provide a suggested update - I could for example in the above case suggest an addition to the .gitlab-ci.yml which overrides image: for the included file job. In the example above it would look like

the-jobs-name:
  image: haynes/jacoco2cobertura:v2.x

[viceice]

@erzz can you please paste some concrete sample pipeline and write down what there should happen.

You can use any fake repos of cause.

[github-actions]

Hi there,

The Renovate team needs your help! To fix the problem, we first need to know exactly what's causing the bug. A minimal reproduction help us to pinpoint the exact cause of the bug.

To get started, please read our guide on minimal reproductions to understand what is needed.

We may close the issue if you have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

[rarkins]

The original reproduction by @j0nm1 is no longer on gitlab.com so this issue is back to status:requirements until someone can create one

[j0nm1]

Hi, totally forgot about this issue. The repository is now public again.

[github-actions]

Thank you for providing a reproduction! 🎉 🚀

The Renovate team will take a look at the reproduction repository. Once we confirm the provided repository reproduces the problem, the label will be changed to reproduction:confirmed.

[erzz]

I'll add a specific reproduction for the case I refer to here as well.

I have started work on a Gitlab CI templates project which we use across various projects. Its public so you can see what kinds of things we are talking about and I am even trying to create good docs in it too :)

But the essentials are - I don't want to write a image scanning job by hand for every project - so I create a template:
https://gitlab.com/ingka/templates/blob/master/image-scan/trivy/image-scan.yml

Then in my various projects I can just include it:

.gitlab-ci.yml

include:
  - remote: 'https://gitlab.com/ingka/templates/raw/master/image-scan/trivy/image-scan.yml'

If all the defaults will work for me, then great! I literally don't need to add another character to my CI. But if I want or need to change something about it I can just specify what I want to override in my .gitlab-ci.yml by just specifying the job name and the properties I want to override.

e.g. To change which stage the job works in I add the following after my include

image-scan:
  stage: my-stage

Gitlab pre-processes at run time - including the various include files and merging everything together. It's really powerful!

So anyway - to be relevant - if in the case of this template - there is a new version 0.13.0 of trivy is released, and the template uses 0.12.0 the suggested diff for my project would be:

+ image-scan:
+   image:
+     name: aquasec/trivy:0.13.0

Thats it. But there are various edge cases.

• Perhaps I already have overrides and we need to be careful not to wipe them out in the process.
• image can also be specified as simply ìmage: aquasec/trivy:0.13.0 - people typically use the more verbose image.name when they also what to override the entrypoint

All of these are valid

image-scan:
  image:
    name: aquasec/trivy:0.13.0

image-scan:
  image: aquasec/trivy:0.13.0

image-scan:
  image:
    name: aquasec/trivy:0.13.0
    entrypoint: ["/path/to/some/script.sh"]

[rarkins]

@erzz the value you show above for include.remote is a non-updateable value, e.g. no version. For the other ones, they look like they should work out of the box with the gitlab-ci, but could potentially be renovated with the regex manager instead of not. I don't see anything that seems like it's the same problem as the topic of this issue. Maybe you can create a Discussion in the meantime and move the above content there and we can merge it back here or link to it if it turns out I'm wrong and it is the same problem.

[erzz]

@rarkins

Renovate does read in the includes - even the remotes and seems to do it very well. It discovers that the included file has an update-able value (the image tag) but then when it comes to trying to create a PR for it - it fails because it is attempting to do a string replace in the local git repo - but of course it doesn't exist.

The logic is to detect "remote:" and know that it will need to instead insert the override in the local .gitlab-ci.yml

[renovate-release]

🎉 This issue has been resolved in version 24.80.2 🎉

The release is available on:

• GitHub release
• 24.80.2

Your semantic-release bot 📦🚀