Bitbucket: "Rate limit for this resource has been exceeded"

[froque]

What Renovate type, platform and version are you using?

Self-hosted using docker image with latest tag.

It is being run with a self hosted Gitlab with daily scheduled CI/CD pipelines.

Describe the bug

Using renovate with bitbucket.org sometimes it triggers the Bitbucket API request limits

Renovate could implement a sleep and retry logic, similar to the Bitbucket Branch Source Jenkins plugin

Relevant debug logs

Click me to see logs

ERROR: Repository has unknown error (repository=repo1)
       "err": {
         "name": "HTTPError",
         "timings": {
           "start": 1617717535966,
           "socket": 1617717535966,
           "lookup": 1617717535966,
           "connect": 1617717536072,
           "secureConnect": 1617717536283,
           "upload": 1617717536283,
           "response": 1617717536467,
           "end": 1617717536467,
           "phases": {
             "wait": 0,
             "dns": 0,
             "tcp": 106,
             "tls": 211,
             "request": 0,
             "firstByte": 184,
             "download": 0,
             "total": 501
           }
         },
         "message": "Response code 429 (Unknown Status Code)",
         "stack": "HTTPError: Response code 429 (Unknown Status Code)\n    at Request.<anonymous> (/usr/src/app/node_modules/got/dist/source/as-promise/index.js:117:42)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (internal/process/task_queues.js:93:5)",
         "options": {
           "headers": {
             "user-agent": "https://github.com/renovatebot/renovate",
             "accept-encoding": "gzip, deflate",
             "authorization": "***********",
             "accept": "application/json"
           },
           "url": "https://api.bitbucket.org/2.0/repositories/repo1/pullrequests?pagelen=50&state=OPEN&state=MERGED&state=DECLINED&state=SUPERSEDED&page=3",
           "username": "",
           "password": "",
           "method": "GET",
           "http2": false
         },
         "response": {
           "statusCode": 429,
           "statusMessage": "Unknown Status Code",
           "body": "Rate limit for this resource has been exceeded",
           "headers": {
             "server": "nginx",
             "vary": "Authorization, cookie, user-context",
             "cache-control": "no-cache, no-store",
             "content-type": "text/plain",
             "x-oauth-scopes": "pullrequest:write",
             "x-b3-traceid": "9f96517de255f5c6",
             "x-dc-location": "ash2",
             "strict-transport-security": "max-age=31536000; includeSubDomains; preload",
             "date": "Tue, 06 Apr 2021 13:58:56 GMT",
             "x-served-by": "app-3018",
             "x-static-version": "8e6a731ff6ee",
             "x-accepted-oauth-scopes": "pullrequest",
             "x-credential-type": "apppassword",
             "x-render-time": "0.0503678321838",
             "connection": "close",
             "x-request-count": "3448",
             "x-frame-options": "SAMEORIGIN",
             "x-version": "8e6a731ff6ee",
             "content-length": "46"
           },
           "httpVersion": "1.1"
         }
       }

Have you created a minimal reproduction repository?

Please read the minimal reproductions documentation to learn how to make a good minimal reproduction repository.

• I have provided a minimal reproduction repository
• I don't have time for that, but it happens in a public repository I have linked to
• I don't have time for that, and cannot share my private repository
• The nature of this bug means it's impossible to reproduce publicly

Additional context

[rarkins]

It would be better if you add a hostRules entry with authentication for bitbucket.org, e.g. like

{
  "domainName": "bitbucket.org",
  "token": "abc123"
}

[froque]

Not everytime, just sometimes.

[rarkins]

And would you like to retry every 5 seconds forever, like the Jenkins code does? It's a pretty crude solution.

[froque]

I agree, but don't see any other solution.

[rarkins]

@viceice @JamieMagee what do you think about a new hostRules field like maxRetryInterval which is intended as an upper bound for retry interval?

For hosts which return a retry-after then Renovate would keep retrying as long as the value is less. For a host which doesn't return any value - such as in this topic - then it will be the default sleep/retry interval. We might also need a maximum retries or maximum retry total

[viceice]

sounds good

[rarkins]

This will require a feature addition: #9550