Any ideas for saving Github Quota when using Probot?

[oscard0m]

Problem

Our Probot app is managing PR Check Runs for ~500 repositories. When Renovate runs because a new npm dependency is published, it opens a Pull Request for each of these repositories.

After that PR creation, our Probot app:

• Creates 6 check_runs for each PR (Build, Deploy Preview, Install Dependencies, JSON Validation, Lint, Tests)
• When each of the checks concludes, updates the check with its conclusion: successful or failure

Considering the app Rate limit it's 12,500, we are very close to get some Github Quota issues when updating internal npm packages before the quota gets refreshed. One solution would be to split in 2 apps this check_run management, one creating the checks and another one updating them.

But a part from that, is there any solution or idea from Probot configuration or interacting with Octokit auth utilities to optimiza quota usage?

Thanks

[gr2m]

The probot app should throttle requests to avoid being rate limitted automatically. If your app is running across multiple processes or in a serverless environment, you can use Redis as a throttle backend. See REDIS_URL: https://probot.github.io/docs/configuration/. Or use options.redisConfig with the Probot constructor

[oscard0m]

The probot app should throttle requests to avoid being rate limitted automatically. If your app is running across multiple processes or in a serverless environment, you can use Redis as a throttle backend. See REDIS_URL: probot.github.io/docs/configuration. Or use options.redisConfig with the Probot constructor

It runs under a serverless environment, yes. Since the rate limit issue is the 12,500 one (https://docs.github.com/en/free-pro-team@latest/developers/apps/rate-limits-for-github-apps#server-to-server-requests), which it's refreshed every hour, the throttle solution would not avoid this issue, right?

[gr2m]

It helps to avoiding hitting the abuse limit. And when hitting the rate limit, it should log a warning and retry the request after the rate limit reset. You might want to move the event handling into long running event handler functions to account for the potential delay

[oscard0m]

It helps to avoiding hitting the abuse limit. And when hitting the rate limit, it should log a warning and retry the request after the rate limit reset. You might want to move the event handling into long running event handler functions to account for the potential delay

Right now we have in mind the abuse rate limit and the rate limit (12500 requests an hour):

• abuse rate limit: Checking Github docs it's not clear which is the limit (like N requests per second or minute). Do you have more details on that? We will go throttling the requests and keep adjusting to wait the minimum but I guess it goes against lambda functions, and we will have to play with a long run event handler as you pointed :)

• rate limit: This is the problem I originally pretended to point at the beginning of the post. There is not an easy way to refresh the token rate for a Github App for this right?

Thanks once again for the support and help ❤️

[gr2m]

* **abuse rate limit:** Checking Github docs it's not clear which is the limit (like `N` requests per second or minute). Do you have more details on that? We will go throttling the requests and keep adjusting to wait the minimum but I guess it goes against lambda functions, and we will have to play with a long run event handler as you pointed :)

There are no static numbers regarding abuse limits, and the rules are not made public and subject to change at any time. There is some information on https://docs.github.com/en/free-pro-team@latest/rest/guides/best-practices-for-integrators#dealing-with-abuse-rate-limits. The throttle plugin makes sure that mutating requests (DELETE, PATCH, POST, PUT) are sent sequentially: one mutating request per second. If the request is triggering notifications, it's one request every 3 seconds. I cannot promise that you won't hit abuse limits at all, but it worked well for me.

rate limit: This is the problem I originally pretended to point at the beginning of the post. There is not an easy way to refresh the token rate for a Github App for this right?

You have no control over when the rate limit gets refreshed. It's variable for GitHub Apps, too. It depends on factors such as the amount of users in your organization, I think. Best to contact support, they will be able to share more details, and might have some tips about how to handle rate limiting that I didn't think of. Maybe there are ways to increase it, they will know: support.github.com/contact