Kubernetes API and helm

[AntonTsaberyaby]

What would you like help with?

I would like help with my configuration

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Gitlab

Please tell us more about your question or problem

Hello guys, I have a goal to update the API versions in Kubernetes manifest files in my repositories. I have decided to use the kubernetes-api datasource for this. However, my manifest files are stored not in plain format but as Helm templates. I have enabled debugging and see the following messages in the log:

Could you please advise on the correct way to use Renovate for updating API versions in the manifests when Helm is used?

Thank you!

Logs (if relevant)

DEBUG: Failed to parse Kubernetes manifest. (repository=infra/k8s/jitsi, packageFile=helm/templates/jigasi/deployment.yaml)
       "err": {
         "name": "YAMLException",
         "reason": "missed comma between flow collection entries",

[rarkins]

You're asserting it's a bug?

[AntonTsaberyaby]

I'm not sure. I don't fully understand how the kubernetes-api datasource should work. Is it meant to update non-templated manifests only, or should it also update Helm manifests? Could what I'm describing be a bug?

[viceice]

I think this can be easily supported by stripping templates before parsing in kubernetes manager as we already do in other managers

[rarkins]

I'm wondering why you selected the bug option, especially if you don't understand it.

[rarkins]

I will answer once it's corrected

[AntonTsaberyaby]

Thank you for bringing this to my attention. I will correct this

[rarkins]

Renovate expects this to be valid yaml which can be parsed.

You could work around this immediately by creating a custom regex manager to parse your files.

Alternatively, please create an example reproduction and we'll see if we can build in a capability to compile or strip out the templates first before parsing.

[AntonTsaberyaby]

Ok. I will prepare an reproduction repo.

[AntonTsaberyaby]

Made a repo https://github.com/AntonTsaberyaby/jitsi-helm/tree/main
Config and description in readme

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

To get started, please read our guide on creating a minimal reproduction.

Good luck,

The Renovate team

[viceice]

• feat(manager/kubernetes): strip go templates before parsing #28816

[rarkins]

Are you suggesting that we stop using a YAML parser?

[AntonTsaberyaby]

I'm not sure I can advise this. What options do we have to solve this problem? Can we run an external utility like helm template to check the template's validity?

[rarkins]

The options are:

• Strip templates and hope we still have valid YAML, or
• Compile templates (and hope it doesn't fail), or
• Rewrite the manager to use bespoke parsing (e.g. regex-based) instead of YAML parsing

Your previous comment seemed to imply the third option above, although I assume you're not volunteering to PR it?

[viceice]

I'll look into the reproduction and maybe I can fix the cause

[viceice]

if i'm enabling json mode then we get to this line: {{ $key }}: {{ tpl $value $ | quote }}

https://github.com/renovate-reproductions/28705-jitsi-helm/blob/f5714dcb3bfff436b60b95c99b982dd8ca373623/templates/common-configmap.yaml#L43

this is of cause no valid yaml 😕 . without json:true we get duplicate key errors.

@zharinov Do you think we can use your good enough parser instead? or is it to hard because of the indention based language?

[AntonTsaberyaby]

The options are:

• Strip templates and hope we still have valid YAML, or
• Compile templates (and hope it doesn't fail), or
• Rewrite the manager to use bespoke parsing (e.g. regex-based) instead of YAML parsing

Your previous comment seemed to imply the third option above, although I assume you're not volunteering to PR it?

@rarkins Thanks for the update. I can try to figure it out, but I don't write in Node.js, so I probably won't be able to make an adequate fix in a reasonable amount of time.

[viceice]

• feat(util/yaml): replace more go templates #29061