Prometheus exporter to monitor the Github API rate-limits.
If you have ever tried using Github's API, you have probably already run into Github's rate limiting errors.
Without authorization, Github only allows 60 requests per hour (!!), and after creating and using your
own personal access token, you are upgraded to 5000 requests per hour.
That rate limit can be easily reached in just a few minutes by calling the API to list basic information from a medium
sized repository and you would have to wait 50+ minutes for the rate limit to reset in order to continue or try again.
You can up the rate limit to 15,000 with an enterprise account but the problem remains the same:
If your service/program needs to make consistent and/or sustained requests to Github's APIs,
you will have to work around and make the most of Github's rate limits.
So, the Prometheus Github rate-limits exporter was introduced to expose the remaining, used, limit
(quotas) and reset (rate-limit window resets in UTC seconds) information per Github API as metrics in order
to create alerting events and (Grafana) monitoring dashboards.
For the exporter to fetch and expose the Github API rate-limits, you need to supply:
Install the tool through pypi:
pip3 install github-rate-limits-exporter
./github-rate-limits-exporter --help
Clone the repository and install tox
:
pip install --user tox
tox -e run-exporter -- \
--github-auth-type pat \
--github-account my_account_name \
--github-token my_token
tox -e run-exporter -- \
--github-auth-type app \
--github-account my_account_name \
--github-app-id my_app_id \
--github-app-installation-id my_installation_id \
--github-app-private-key-path /path/to/private_key.pem
Clone the repositroy and build the docker container image:
docker build -t prometheus-gh-rate-limit-exporter:latest .
or pull directly from Docker Hub:
docker pull theodore86/prometheus-gh-rate-limit-exporter:latest
docker run -p 10050:10050 -d \
--name gh-rl-exporter \
--restart on-failure \
-e GITHUB_ACCOUNT=my_account_name \
-e GITHUB_AUTH_TYPE=pat \
-e GITHUB_TOKEN=my_token \
-e EXPORTER_LOG_LEVEL=4 \
prometheus-gh-rate-limit-exporter:latest
docker run -p 10050:10050 -d \
--name gh-rl-exporter \
--restart on-failure \
-e GITHUB_AUTH_TYPE=app \
-e GITHUB_ACCOUNT=my_account_name \
-e GITHUB_APP_ID=111111 \
-e GITHUB_APP_INSTALLATION_ID=22222222 \
-e GITHUB_APP_PRIVATE_KEY_PATH=/app/key.pem \
-e EXPORTER_LOG_LEVEL=4 \
--mount type=bind,source=/ws/key.pem,target=/app/key.pem,readonly \
prometheus-gh-rate-limit-exporter:latest
Another way to test and observe the exporter is through the Prometheus/Grafana/AlertManager
stack on top of the docker-compose:
Clone the repository and install tox
tool:
pip install --user tox
export GITHUB_AUTH_TYPE=pat
export GITHUB_TOKEN=your_token
export GITHUB_ACCOUNT=your_account
export EXPORTER_LOG_LEVEL=4
export GF_SECURITY_ADMIN_USER=username
export GF_SECURITY_PASSWORD=password
tox -e dc-run
export GITHUB_ACCOUNT=your_account
export GITHUB_AUTH_TYPE=app
export GITHUB_APP_ID=12345
export GITHUB_APP_INSTALLATION_ID=123456
export GITHUB_APP_SRC_PRIVATE_KEY_PATH=/ws/private_key.pem
export GITHUB_APP_PRIVATE_KEY_PATH=/tmp/private_key.pem
export EXPORTER_LOG_LEVEL=4
export GF_SECURITY_ADMIN_USER=username
export GF_SECURITY_PASSWORD=password
tox -e dc-run
- Portainer (credentials: admin/adminpassword)
See also the existing examples:
tox -e dc-clean
This helm chart helps to install and configure github-rate-limits exporter on Kubernetes clusters.
-
Install:
curl -fsSL https://get.jetify.com/devbox | bash
- Create the cluster:
cd kubernetes; devbox run start-cluster
- Adjust the environment file (.env):
vim .env
- Bring up the github-rate-limits exporter and the Kubernetes Prometheus stack:
devbox run --env-file .env start-monitor
devbox run --env-file .env stop-monitor
devbox run stop-cluster