GitLab CI manager doesn't support usage of environment variables in includes

[peschmae]

What would you like help with?

I think I found a bug

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

GitLab 16.9, Renovate 37.250

Please tell us more about your question or problem

We started to use GitLab ci components just recently, and realized that renovate bot is supporting updating those as well.

In the official documentation GitLab references the GitLab server using the $CI_SERVER_FQDN variabel (eg: https://docs.gitlab.com/ee/ci/components/#replace-hardcoded-values-with-inputs ), we assumed this to be best practice, and did the same thing.

When we wanted to use renovate to update our components, we didn't get any pull requests, and saw in the logs that the authentication with the GitLab server failed, when trying to lookup the components version (see logs below).

Digging into it, we realized that the $CI_SERVER_FQDN variable isn't resolved, and the variable name used as a server name.

I think adding a hint in the documentation that the environment variables aren't resolved, might help other solve the issue faster, while at the same time avoiding any security issues, that might arise if renovate would resolve the environment variable if present in the string.

Logs (if relevant)

Logs

DEBUG: hostRules: no authentication for $ci_server_fqdn (repository=tools/policy_reporter)
DEBUG: GET https://$CI_SERVER_FQDN/api/v4/projects/helpers%2Fci-components/repository/tags?per_page=100 = (code=ECONNRESET, statusCode=-1 retryCount=2, duration=3434) (repository=tools/policy_reporter)
DEBUG: Gitlab API error (repository=tools/policy_reporter)
       "err": {
         "name": "RequestError",
         "code": "ECONNRESET",
         "timings": {
           "start": 1710603221740,
           "socket": 1710603225173,
           "lookup": 1710603225173,
           "connect": 1710603225173,
           "error": 1710603225174,
           "phases": {"wait": 3433, "dns": 0, "tcp": 0, "total": 3434}
         },
         "message": "Client network socket disconnected before secure TLS connection was established",
         "stack": "RequestError: Client network socket disconnected before secure TLS connection was established\n    at ClientRequest.<anonymous> (/usr/local/renovate/node_modules/.pnpm/got@11.8.6/node_modules/got/dist/source/core/index.js:970:111)\n    at Object.onceWrapper (node:events:632:26)\n    at ClientRequest.emit (node:events:529:35)\n    at ClientRequest.emit (node:domain:489:12)\n    at ClientRequest.origin.emit (/usr/local/renovate/node_modules/.pnpm/@szmarczak+http-timer@4.0.6/node_modules/@szmarczak/http-timer/dist/source/index.js:43:20)\n    at TLSSocket.socketErrorListener (node:_http_client:501:9)\n    at TLSSocket.emit (node:events:529:35)\n    at TLSSocket.emit (node:domain:489:12)\n    at emitErrorNT (node:internal/streams/destroy:151:8)\n    at emitErrorCloseNT (node:internal/streams/destroy:116:3)\n    at connResetException (node:internal/errors:720:14)\n    at TLSSocket.onConnectEnd (node:_tls_wrap:1714:19)\n    at TLSSocket.emit (node:events:529:35)\n    at TLSSocket.emit (node:domain:489:12)\n    at endReadableNT (node:internal/streams/readable:1400:12)\n    at processTicksAndRejections (node:internal/process/task_queues:82:21)",
         "options": {
           "headers": {
             "user-agent": "RenovateBot/37.250.0 (https://github.com/renovatebot/renovate)",
             "accept": "application/json",
             "accept-encoding": "gzip, deflate, br"
           },
           "url": "https://$ci_server_fqdn/api/v4/projects/helpers%2Fci-components/repository/tags?per_page=100",
           "hostType": "gitlab-tags",
           "username": "",
           "password": "",
           "method": "GET",
           "http2": false
         }
       }

[rarkins]

In your example how does this variable value compare to the endpoint you configure into Renovate?

[peschmae]

We are settings the endpoint using the RENOVATE_ENDPOINT env variable, which references the CI_API_V4_URL environment variable.

CI_API_V4_URL is like : https://gitlab.example.biz/api/v4/
and CI_SERVER_FQDN is gitlab.example.biz

[peschmae]

In the logs I attached, I didn't change the FQDNs shown, it really is just the string "$CI_SERVER_FQDN" (sometimes uppercase, sometimes lowercase), that renovate tries to use for the Server URL that it connects to

[rarkins]

Example from GitLab docs:

stages: [verify, deploy]

include:
  - component: $CI_SERVER_FQDN/myorg/ruby/test@1.0.0
    inputs:
      stage: verify

@viceice @secustor do you think there's any problem if we do env substitution for extraction in this case?

[secustor]

We are already resolving the Dependency proxy env variables in GitlabCI files, so there isn't much of an additional attack surface.

[viceice]

we should mention that this variable is taken from the renovate runner repo and not from the processed repo, just in case there a difference.

[secustor]

I have been thinking more in the direction of using endpoint and detectPlatform or something similar for that purpose, rather than looping env variables from the runner environment.

[candrews]

I think it would great for the GitLab CI manager to support registry alias so I've created a PR adding that support: #28607

With that improvement, configuration like the following in Renovate's config.js (following the documentation at https://docs.renovatebot.com/configuration-options/#registryaliases), addresses this need:

{
  "registryAliases": {
    "$CI_SERVER_FQDN": process.env.CI_SERVER_FQDN,
    "$CI_SERVER_HOST": process.env.CI_SERVER_HOST
  }
}