File with umlaut ü (special character) in its name gets corruped

[cypressious]

How are you running Renovate?

Mend Renovate hosted app on github.com

If you're self-hosting Renovate, tell us what version of Renovate you run.

No response

If you're self-hosting Renovate, select which platform you are using.

No response

If you're self-hosting Renovate, tell us what version of the platform you run.

No response

Was this something which used to work for you, and then stopped?

It used to work, and then stopped

Describe the bug

My repo contains a file with an umlaut ü in its name. In the PRs that renovate creates, the files get corrupted.

Example: cypressious/RenovateUmlautIssue#3

Relevant debug logs

Logs

Copy/paste the relevant log(s) here, between the starting and ending backticks

Have you created a minimal reproduction repository?

I have linked to a minimal reproduction repository in the bug description

[Gabriel-Ladzaretti]

Confirmed when running with the app.
although this does not happen when running self hosted (docker on 34.20.4).

[Gabriel-Ladzaretti]

might be a git problem ?

git config --global core.quotePath false might help

https://git-scm.com/docs/git-config#Documentation/git-config.txt-corequotePath

https://stackoverflow.com/questions/34549040/git-not-displaying-unicode-file-names

https://stackoverflow.com/questions/22827239/how-to-make-git-properly-display-utf-8-encoded-pathnames-in-the-console-window

[rarkins]

So it works fine if running within the renovate/renovate container? I wasn't aware we'd taken any steps to make that work, I'd like to understand why it is

[Gabriel-Ladzaretti]

here are both PRs
Self hosted - expected (1 file changed)

• Update dependency Newtonsoft.Json to v13 ladzaretti/RenovateUmlautIssue#10

App - two affected files

• prefix_from_script Update dependency Newtonsoft.Json to v13 ladzaretti/RenovateUmlautIssue#7

[rarkins]

Are the branch logs essentially identical?

[Gabriel-Ladzaretti]

both runs deliberately committing 1 file

[Gabriel-Ladzaretti]

This might be caused by an environment/git version combination (?)
see https://stackoverflow.com/questions/34549040/git-not-displaying-unicode-file-names

In the renovate docker container we have -

[viceice]

@rarkins maybe the hosted app sets some explicit LC or Lang variables beside the default containerbase?

[rarkins]

The hosted app uses renovate/docker-node. Right now:

user@9a8c5ee11c73:/$ git --version
git version 2.37.3
user@9a8c5ee11c73:/$ env | grep LANG
LANG=C.UTF-8
user@9a8c5ee11c73:/$ env | grep LC
LC_ALL=C.UTF-8

[cypressious]

Any updates on this?

[rarkins]

The renovate/renovate:slim image has the same language settings as renovate/node-docker:

ubuntu@90b4f3fdb06a:/usr/src/app$ env | grep LC
LC_ALL=C.UTF-8
ubuntu@90b4f3fdb06a:/usr/src/app$ env | grep LANG
LANG=C.UTF-8
ubuntu@90b4f3fdb06a:/usr/src/app$

I will try upgrading git but if that doesn't fix it then I'm out of ideas.

[rarkins]

FYI the production containers now have git 2.38.1

[cypressious]

https://github.com/cypressious/RenovateUmlautIssue/pull/3/files still looks broken after retrying the PR.

[rarkins]

Reproduction confirmed in https://github.com/renovate-reproductions/18805/pull/1/files

[rarkins]

DEBUG: Updated 1 package files(branch="renovate/newtonsoft.json-12.0.x")
DEBUG: No updated lock files in branch(branch="renovate/newtonsoft.json-12.0.x")
DEBUG: 1 file(s) to commit(branch="renovate/newtonsoft.json-12.0.x")
DEBUG: Preparing files for committing to branch renovate/newtonsoft.json-12.0.x(branch="renovate/newtonsoft.json-12.0.x")
DEBUG: Setting git author name: Renovate Bot(branch="renovate/newtonsoft.json-12.0.x")
DEBUG: Setting git author email: bot@renovateapp.com(branch="renovate/newtonsoft.json-12.0.x")
DEBUG: git commit(branch="renovate/newtonsoft.json-12.0.x")
{
  "deletedFiles": [],
  "ignoredFiles": [],
  "result": {
    "author": null,
    "branch": "renovate/newtonsoft.json-12.0.x",
    "commit": "697aaa6266ad48d3d252bae684625a6a784b6e08",
    "root": false,
    "summary": {
      "changes": 1,
      "insertions": 1,
      "deletions": 1
    }
  }
}

[rarkins]

I bet the difference/problem is related to platformCommit, which is enabled in the app. @zharinov can you think about it?

[zharinov]

This is how we get this into our system:

❯ git cat-file -p e37acd41a1d026cccdff5f3a457025c9dc5cdda0
100644 blob add57be707d1a627fd960286263733b8f2df2dcb    .gitignore
040000 tree 784ed96879c476ac5fa4cea6d5e4124b7d3db0a5    .idea
040000 tree c2ef867f66d41d517be0dce7ccc1456c59666cba    ConsoleApp1
100644 blob e588b00a65fc0cc1306e7e1ae4be494aedb61a42    "Sol\303\274tion.sln"
100644 blob 2c63c0851048d8f7bff41ecf0f8cee05f52fd120    renovate.json

I have no idea why this weird encoding takes place, and yet don't know how to fix it

UPD. It's core.quotePath Git option is true by default

[cypressious]

When will the fix be deployed? I'm not seeing it in my reproduction example above, yet.

[rarkins]

I put the new version live just now but unfortunately it does not seem to have fixed the problem when I try with my reproduction repo.

FYI @zharinov

[rarkins]

INFO: Repository started
{
  "renovateVersion": "34.27.3"
}

DEBUG: 1 file(s) to commit(branch="renovate/newtonsoft.json-12.0.x")
DEBUG: Preparing files for committing to branch renovate/newtonsoft.json-12.0.x(branch="renovate/newtonsoft.json-12.0.x")
DEBUG: Setting git author name: Renovate Bot(branch="renovate/newtonsoft.json-12.0.x")
DEBUG: Setting git author email: bot@renovateapp.com(branch="renovate/newtonsoft.json-12.0.x")
DEBUG: git commit(branch="renovate/newtonsoft.json-12.0.x")
{
  "deletedFiles": [],
  "ignoredFiles": [],
  "result": {
    "author": null,
    "branch": "renovate/newtonsoft.json-12.0.x",
    "commit": "b60804ffdb211e9cd0cb045d5d30cc0cfc172459",
    "root": false,
    "summary": {
      "changes": 1,
      "insertions": 1,
      "deletions": 1
    }
  }
}

[zharinov]

@rarkins Can't this be caused by one of our caching mechanisms?

[rarkins]

Based on the log, I don't think so?

[rarkins]

I closed/deleted the existing PR and its replacement is correct: renovate-reproductions/18805#4

So unless there's a simple way for us to fix existing PRs, I'd like to still close this as it shouldn't happen for any new PRs.

[zharinov]

I think rebase/retry checkbox should be enough to fix it for any particular existing PR

[cypressious]

Rebase/retry checkbox worked now, thank you so much 🙂