respect cloneSubmodules as same as forkProcessing

[childnode]

Tell us more.

yes, we had an bug for that ;)

#8936 from @andreibsk

but stalled and being closed by bot

---

Suggestion: allow cloneSubmodules to be respected as like forkProcessing in onboardingConfigFileName

Issue to solve: clone of repository and submodule processing is made before project config is read! That leads to the case, that a private submodule (from sight of renovate on a self-hosted instance) can't be cloned and repository processing fails.
The project config cloneSubmodules: false is being ignored

Proposal to fix: as like forkProcessing, the config might be altered in https://github.com/renovatebot/renovate/blob/main/lib/workers/repository/init/apis.ts

to be checked: https://github.com/renovatebot/renovate/blob/main/lib/workers/repository/init/index.ts#L69 is being processed after config merge so there shouldn't be an issue at all, but rundown says.

DEBUG: Repo is onboarded (repository=my/repo)
DEBUG: Initializing git repository into /tmp/renovate/repos/gitlab/my/repo (repository=my/repo)
DEBUG: Performing blobless clone (repository=my/repo)
DEBUG: git clone completed (repository=my/repo)
       "durationMs": 1720
DEBUG: Cloning git submodule at secret-submodule
>>>>> ERROR <<<<<<<
DEBUG: Found .renovaterc.json config file
DEBUG: Repository config (repository=my/repo)
       "fileName": ".renovaterc.json",
       "config": {
         "$schema": "https://docs.renovatebot.com/renovate-schema.json",
         "cloneSubmodules": false
       }
…
DEBUG: No vulnerability alerts found
…
DEBUG: Git function thrown …
DEBUG: Repository result: temporary-error, status: onboarded, enabled: true, onboarded: true (repository=my/repo)

perhaps the forkProcessing Feature itself corrupted this by initRepo ! https://github.com/renovatebot/renovate/blob/main/lib/workers/repository/init/apis.ts#L97

---

ref behavior of forkProcessing as documented:

Allowed filenames

Only the onboardingConfigFileName (which defaults to renovate.json) is supported for forkProcessing. You can't use other > filenames because Renovate only checks the default filename when using the Git-hosting platform's API.

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

To get started, please read our guide on creating a minimal reproduction.

Good luck,

The Renovate team

[childnode]

to reproduce:

1. Create a public repository test
2. create a private repostory sub
3. cd test && git submodule add sub ../sub
4. get access token for test
5. run renovate on test

alter cases by

• ✅ cloneSubmodules: false in renovate global config only: succeeds dep extraction ✅
• ❌ cloneSubmodules: true in renovate global config only: fails dep extraction
• ❌ cloneSubmodules: true in renovate global config but cloneSubmodules: false in project config: fails dep extraction
• ✅ cloneSubmodules: false in renovate global config but cloneSubmodules: true in project config: fails dep extraction for git-submodule depmanager only

[github-actions[bot]]

Hi there,

This issue or discussion is missing some logs, making it difficult or impossible to help you. Depending on which situation applies follow one, some or all of these instructions.

No logs at all

If you haven't posted any log yet, we need you to find and copy/paste the log into the issue template.

Finding logs on hosted app

Select me to read instructions

If you use the Mend Renovate app (GitHub):

1. Log in to the Mend Developer Portal
2. Navigate to the correct organization and repository
3. Locate the appropriate log (it may not always be the latest one)
4. Copy/paste the log contents
5. Follow the steps in the formatting your logs section

Finding logs when self-hosting

Select me to read instructions

If you're running self-hosted, run with LOG_LEVEL=debug in your environment variables and search for whatever dependency/branch/PR that is causing the problem.

Insufficient logs

Select me to read instructions

If you already gave us a log, and the Renovate team said it's not enough, then follow the instructions from the No logs at all section.

Formatting your logs

Select me to read instructions

Please put your logs in a <details> and <summary> element like this:

<details><summary>Select me to see logs</summary>

```
Copy/paste your logs here, between the starting and ending backticks
```

</details>

If you feel the logs are too large to paste here, please use a service like GitHub Gist and paste the link here.

Good luck,

The Renovate team

[childnode]

as long it is not onboarded or project config has cloneSubmodules: false dependency extraction completes

DEBUG: git clone completed (repository=childnode/test)
       "durationMs": 802
DEBUG: latest repository commit (repository=childnode/test)
       "latestCommit": {
         "hash": "ff2d4e4332f9aba49ecf4792e7988e0ea989ff8a",
         "date": "2024-04-10T08:53:39+02:00",
         "message": "init",
         "refs": "HEAD -> main, origin/main, origin/HEAD",
         "body": "",
         "author_name": "childnode",
         "author_email": "redacted@example.com"
       }
DEBUG: No renovate config file found (repository=childnode/test)
…
DEBUG: Matched 1 file(s) for manager git-submodules: .gitmodules (repository=childnode/test)
 WARN: Error mapping git submodules during extraction (repository=childnode/test, packageFile=.gitmodules)
       "err": {
         "task": {
           "commands": [
             "ls-remote",
             "--symref",
             "https://git.instance.local/childnode/sub",
             "HEAD"
           ],
           "format": "utf-8",
           "parser": "[function]"
         },
         "message": "remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://git.instance.local/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied\nfatal: Authentication failed for 'https://git.instance.local/childnode/sub/'\n",
         "stack": "Error: remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://**redacted**@3.24.0/node_modules/simple-git/src/lib/plugins/error-detection.plugin.ts:42:29)\n    at PluginStore.exec (/usr/local/renovate/node_modules/.pnpm/simple-git@3.24.0/node_modules/simple-git/src/lib/plugins/plugin-store.ts:54:29)\n    at /usr/local/renovate/node_modules/.pnpm/simple-git@3.24.0/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:124:42\n    at new Promise (<anonymous>)\n    at GitExecutorChain.handleTaskData (/usr/local/renovate/node_modules/.pnpm/simple-git@3.24.0/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:121:14)\n    at GitExecutorChain.<anonymous> (/usr/local/renovate/node_modules/.pnpm/simple-git@3.24.0/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:97:40)\n    at Generator.next (<anonymous>)\n    at fulfilled (/usr/local/renovate/node_modules/.pnpm/simple-git@3.24.0/node_modules/simple-git/dist/cjs/index.js:52:24)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)"
       }
DEBUG: manager extract durations (ms) (repository=childnode/test)
       "managers": {"git-submodules": 366}
DEBUG: Found git-submodules package files (repository=childnode/test)
DEBUG: Found 1 package file(s) (repository=childnode/test)
 INFO: Dependency extraction complete (repository=childnode/test, baseBranch=main)
       "stats": {
         "managers": {"git-submodules": {"fileCount": 1, "depCount": 0}},
         "total": {"fileCount": 1, "depCount": 0}
       }

but when you enable global level cloneSubmodules: true clone fails unexpected

EBUG: git clone completed (repository=childnode/test)
       "durationMs": 899
DEBUG: latest repository commit (repository=childnode/test)
       "latestCommit": {
         "hash": "27279aa11666ebb48888876dcb710e513f8b1087",
         "date": "2024-04-10T07:11:40+00:00",
         "message": "added \"cloneSubmodules\": true,",
         "refs": "HEAD -> main, origin/main, origin/HEAD",
         "body": "",
         "author_name": "childnode",
         "author_email": "redacted@example.com"
       }
DEBUG: Found renovate.json config file (repository=childnode/test)
DEBUG: Repository config (repository=childnode/test)
       "fileName": "renovate.json",
       "config": {
         "$schema": "https://docs.renovatebot.com/renovate-schema.json",
         "cloneSubmodules": true
       }
…
DEBUG: Git function thrown (repository=childnode/test)
       "err": {
         "task": {
           "commands": ["submodule", "update", "--init", "sub"],
           "format": "utf-8",
           "parser": "[function]"
         },
         "message": "Submodule 'sub' (https://**redacted**@git.instance.local/childnode/sub) registered for path 'sub'\nCloning into '/tmp/renovate/repos/gitlab/childnode/test/sub'...\nremote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://**redacted**@git.instance.local/childnode/sub' into submodule path '/tmp/renovate/repos/gitlab/childnode/test/sub' failed\nFailed to clone 'sub'. Retry scheduled\nCloning into '/tmp/renovate/repos/gitlab/childnode/test/sub'...\nremote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://**redacted**@git.instance.local/childnode/sub' into submodule path '/tmp/renovate/repos/gitlab/childnode/test/sub' failed\nFailed to clone 'sub' a second time, aborting\n",
…
DEBUG: Git function thrown (repository=childnode/test)
       "err": {
         "task": {
           "commands": ["checkout", "-f", "--recurse-submodules", "main", "--"],
           "format": "utf-8",
           "parser": "[function]"
         },
         "message": "fatal: not a git repository: ../.git/modules/sub\nfatal: could not reset submodule index\n",
…
DEBUG: Repository result: temporary-error, status: onboarded, enabled: true, onboarded: true (repository=childnode/test)

Only If global setting is cloneSubmodules: false but only project level is cloneSubmodules: true depmanager fails as expected:

 WARN: Error mapping git submodules during extraction (repository=childnode/test, packageFile=.gitmodules)
       "err": {
         "task": {
           "commands": [
             "ls-remote",
             "--symref",
             "https://git.instance.local/childnode/sub",
             "HEAD"
           ],
           "format": "utf-8",
           "parser": "[function]"
         },
         "message": "remote: HTTP Basic: Access denied. …'\n",
…
       }
DEBUG: manager extract durations (ms) (repository=childnode/test)
       "managers": {"git-submodules": 366}
DEBUG: Found git-submodules package files (repository=childnode/test)
DEBUG: Found 1 package file(s) (repository=childnode/test)
 INFO: Dependency extraction complete (repository=childnode/test, baseBranch=main)
       "stats": {
         "managers": {"git-submodules": {"fileCount": 1, "depCount": 0}},
         "total": {"fileCount": 1, "depCount": 0}
       }