How to debug regexManagers?

[dbanty]

How are you running Renovate?

Mend Renovate hosted app on github.com

If you're self-hosting Renovate, tell us what version of Renovate you run.

No response

If you're self-hosting Renovate, select which platform you are using.

None

If you're self-hosting Renovate, tell us what version of the platform you run.

No response

Was this something which used to work for you, and then stopped?

I never saw this working

Wanted end result.

In general, I want to know how to debug a regexManager. When looking at logs on https://app.renovatebot.com, I see no indication that my regexManager was run, nor any error messages about what's wrong with it.

For more info, I want to auto-update a string within my GitHub workflows:

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
          token: ${{ secrets.PAT }}
      - name: Install Knope
        uses: knope-dev/action@v1
        with:
          version: 0.6.2
      - run: knope release --dry-run

That version: 0.6.2 defines the version of the tool which should be auto-updated. It is a crate called knope.

What you tried so far.

This is my Renovate config:

{
  "extends": [
    "config:base",
    ":semanticCommitTypeAll(chore)"
  ],
  "rangeStrategy": "widen",
  "regexManagers": [
    {
      "fileMatch": ["release.*\\.yml"],
      "matchStrings": ["version:\\s*(?<currentValue>.*?)"],
      "depNameTemplate": "knope",
      "datasourceTemplate": "crate"
    }
  ]
}

I've tried several variations on fileMatch and matchStrings, none seem to give any feedback in the Renovate logs.

Relevant debug logs

I haven't found any relevant logs, which is the main problem. I don't even know how to diagnose 😓. Here is a gist containing the full output of one run though: https://gist.github.com/dbanty/093db02c482add9f7c2fbc89e6b61d93

[dbanty]

I fixed the specific bug—my regex was lazy. I still think it'd be worth knowing how to get more verbose info out of Renovate to find out what it's doing with the regex manager. For example, it getting a version of "" should be an error, I'd think.

[reitzig]

+1000

In your case, the log mentions applying regex managers at dbanty/093db02c482add9f7c2fbc89e6b61d93#file-log-txt-L331 and then never mentions it again.

I have the same problem with this "beauty":

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "regexManagers": [
    {
      "description": "Detect all image names of the form 'someNameImage : \"registry/image:tag@sha256:hash\"'",
      "fileMatch": ["^vars/globals.groovy"],
      "matchStrings": [
          "^\\s*\\w+Image\\s*:\\s*\"(?:(?<registryUrl>[^/]+)/)?(?<depName>[^\":@]+)(?::(?<currentValue>[^\"@]+))?(?:@(?<currentDigest>sha256:[^\"]+))?\""
      ],
      "datasourceTemplate": "docker",
      "versioningTemplate": "docker"
    }
  ]
}

Snippet of the file:

def call() {
    def ecrHostname = '123456.dkr.ecr.eu-central-x.amazonaws.com'
    def dockerHubProxy = 'docker-proxy.some.org'
    vars = [
            //credentials
            bitbucketSsh                  : 'jenkins-bitbucket-ssh',
            bitbucketCreds                : 'jenkins-bitbucket-credentials',
 
            helmAgentImage                : "${ecrHostname}/shared-tools/helm-agent",
            kanikoAgentImage              : "${ecrHostname}/shared-tools/kaniko-agent",
            renovateConfigValidatorImage  : "${dockerHubProxy}/renovate/renovate:slim",
            asciidoctorAgentImage         : "${dockerHubProxy}/asciidoctor/docker-asciidoctor:latest",
            ruleTestImage                 : "docker-proxy.some.org/asciidoctor/docker-asciidoctor:latest",

            // paths
            kubeconfig                    : './',
            chartPath                     : './charts/*',
            docsTarget                    : 'docs/target',
            docsSource                    : './docs/src/main/asciidoc'
    ]

    return vars
}

I get, in a lookup dry-run on a "base" branch:

DEBUG: Matched 1 file(s) for manager regex: vars/globals.groovy (repository=cc/jenkins_shared_libraries, baseBranch=feat/renovate-image-refs-in-globals)
DEBUG: manager extract durations (ms) (repository=cc/jenkins_shared_libraries, baseBranch=feat/renovate-image-refs-in-globals)
       "managers": {"pre-commit": 0, "maven": 15, "regex": 15}
DEBUG: Found maven package files (repository=cc/jenkins_shared_libraries, baseBranch=feat/renovate-image-refs-in-globals)
DEBUG: Found 1 package file(s) (repository=cc/jenkins_shared_libraries, baseBranch=feat/renovate-image-refs-in-globals)
 INFO: Dependency extraction complete (repository=cc/jenkins_shared_libraries, baseBranch=feat/renovate-image-refs-in-globals)
       "stats": {
         "managers": {"maven": {"fileCount": 1, "depCount": 12}},
         "total": {"fileCount": 1, "depCount": 12}
       }

Clearly, the (single) regex manager is being applied to the file -- but then, silently, no dependencies are found.

Best I can determine, the regex is syntactically fine and matches the relevant lines. I did the best with the matching groups and properties I could glean from the docs (I have gotten regex managers for image references to work before), but clearly there's a mistake. Sadly, Renovate doesn't tell us what.

FWIW, I don't expect registries of the form ${...} to work -- but I'd expect seeing 404s (or some other error) before tackling that with depNameTemplate or registryUrlTemplate (with replace?). I'll note that I included the fully qualified ruleTestImage specifically to check for this; not even that line is being picked up as expected.

[HonkingGoose]

Thank you very much for listing your problems! ❤️

@rarkins please read the list of "feature requests" and let me know if you want me to create issues for the feature requests you like? 😉

[HonkingGoose]

@rarkins I'm willing to break these "todos" out into their own issues, but I first need to know which todo's you'll accept issues for. 😉

[HonkingGoose]

I found this issue to allow Renovate to self-test basic functionality:

• feat: diagnostic option #4403

[rarkins]

Ideally, there would be a tool I can run locally on my repo checkout that performs the equivalent of extract and, optionally, lookup dry-runs. Having to wait for the CI system to run the dry runs is what makes all of this super annoying.

#3609 - Support local git/clone run

As for the regex managers, I honestly don't know what to change in the docs. Sure, the how-to could be improved -- currently, relevant information is spread across multiple places. But if you read it all, you have all the relevant information. Well, had. We all know that even great docs don't replace good errors, don't we? 😉

The biggest challenge with regex managers is regex. When regex doesn't match.. it just doesn't. It's not like we can log every single time a regex doesn't match a string in any file because it would be thousands or more.

• Align API with packageRules. fileMatch uses regex but matchPaths glob? They often come in matching pairs, and having to use different syntax is confusing.

#21737 - Consistency of blob vs regex across different configuration options

• Clarify packageName vs depName. They seem to be used differently, sometimes interchangeably.

We could add this to docs. For now this issue describes a little: #16012

• Clarify the interface between managers and data sources. Currently, you need to know what the data source expects while configuring the manager; different sources require, admit, and interpret some values differently! (Far as I can tell, anyway.)

Needs more specifics. I suspect just that a few datasources could be better described when they're not so obvious, e.g. git-tags

• Let me give each regex manager a (unique) name. Log output that aggregates 7 regex managers doesn't help me; I want to see what jib-parent-image did on pom.xml!

I like this idea, maybe in conjunction with us moving to the term "custom" managers to allow more than just regex.

#21760 - Allow naming of regex managers

• Log levels are all over the place.

  • INFO is useless for debugging (doesn't even contain the results of each "stage"),

This is intentional

and DEBUG can be far too detailed (HTTP statistics?)

I don't agree with "far too detailed". Knowing the exact URLs called has proven useful many times for "debugging".

• Things I consider errors for config debugging are logged under DEBUG, probably because that's our perspective for a "big" Renovate run.

We suppress some errors into debug because we don't want to trouble administrators. But maybe we should distinguish between "user" errors versus "bot" errors so that administrators can focus on the latter only if desired.

Related: #8646 - Semantic error messages

• I need a "strict"/"config development" mode that logs config errors as ERROR and stops.

That's meant to be our default behavior (config error issue, run is aborted).

• I need to dry-run against config only from a "base" branch, without merging with the default branch.

Please describe this in more detail. You mean for testing a new config? And if so, then wouldn't the local dry run discussed earlier be sufficient?

• It would be helpful to get the JSON of config merge, extraction, lookup, and branch summary, respectively, as plain (or file) output.

There's already an option printConfig to dump the config.

packageFiles with updates covers extraction and lookup results.

• One might even attempt to write unit tests for Renovate configs! (Which I would love to do for presets. That needs more ingredients, though, once we enter lookup territory.)
• Unit tests for regex managers, as one of the more complex pieces of Renovate config, would be a great start.

I don't understand what you mean by these. Within the Renovate code base, or users write their own tests against their own config?

• extract should tell me for each regexManager and (matched) file what matches there were, what the match groups were, and what the values after templating were, as they would be passed to the data source.

Earlier you said debug logging is "far too detailed", but this for me would fall under that. Sounds like something you wouldn't want to log every time, and instead only during config creation. Therefore better suited to trace logging locally than debug logging everywhere

• extract should report any and all problems with templating and "calling" the data source.

Need more details. Extract phase doesn't call datasource

• lookup needs to log any problems; currently, we sometimes only get a short buzzword in skipReason.
• (in "strict" mode), lookup should fail whenever registry access fails. My templating may have produced something syntactically fine that is complete bogus.

We could enable a mode where people opt into more "strict" lookups, meaning fail and log on any lookup error. You'd be surprised how many people want the opposite though (just keep going, try your best..)

Please start a new discussion on this topic (strict, more verbose logging of lookup errors), then once we have reached an agreement we can turn it into a feature request issue.

[reitzig]

Oh boy, it's been a while. My bad. Thanks for your responses!

extract should tell me for each regexManager and (matched) file what matches there were, what the match groups were, and what the values after templating were, as they would be passed to the data source.

Earlier you said debug logging is "far too detailed", but this for me would fall under that. Sounds like something you wouldn't want to log every time, and instead only during config creation. Therefore better suited to trace logging locally than debug logging everywhere

Right. We don't want that during production runs. 🤔

Extract phase doesn't call datasource

Yup. I remember needing lookup to reproduce some problems -- e.g. wrong use of matching groups -- but that was nigh impossible to get to run locally.

I would have loved to run lookup up to executing requests, but then just logging the URLs. At that point, I can confirm whether I fed the datasource in a wrong way.

---

Re #3609: I remember trying the new mode in the context of private infra behind VPN and authentication and immediately rejecting it:

• Shared presets didn't resolve, causing errors (IIRC). Manual workaround possible by removing the extends or copy-paste, but annoying.
• lookup would be tripped up by network connections to yet other protected systems (ECR registry, Nexus, ...)

Maybe I could have gotten all these sorted out, but at that point I fell back to pushing N+1 times to Jenkins. :/

---

In summary, I think what I really would like to have is a modified dry-runner for local usage that

• uses the currently checked out config only (no config merge, no extends),
• runs extract and lookup but
• stubs any requests to data sources with logging the request, and
• outputs otherwise excessive detail about custom managers (and data sources, for that matter).

With that tool, I would be able to iterate rapidly on custom managers using my local dev machine.

For debugging the regexes themselves, I guess a separate command of the form

renovate-regex-tester [--config renovate.json] <custom matcher name> <file>

that runs the exact same matching logic against exactly one file would be useful. Yes, that's basically the same as using regex101.com -- but it would remove any doubts about escaping, which keeps tripping me up. 😩

[jackson-chris]

+1 I stumbled across this after trying to debug my own regexManagers configuration and ran into a lot of the exact same issues that @reitzig has already articulated. It would be great if some of these issues that were opened up in response were prioritized.

[rarkins]

PRs welcome