Bug: Hourly GitHub HTTP API quota exhausted

[DeaMariaLeon]

I have seen this a few times. And trying (without waiting at all) things go back to normal (I can post data to the conbench server).

231103-19:32:13.471 ERROR: Response content: unexpected exception, please report this:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/views.py", line 107, in view
    return current_app.ensure_sync(self.dispatch_request)(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/views.py", line 188, in dispatch_request
    return current_app.ensure_sync(meth)(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask_login/utils.py", line 290, in decorated_view
    return current_app.ensure_sync(func)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/conbench/api/results.py", line 292, in post
    benchmark_result = BenchmarkResult.create(data)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/conbench/entities/benchmark_result.py", line 263, in create
    commit = commit_fetch_info_and_create_in_db_if_not_exists(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/conbench/entities/benchmark_result.py", line 1068, in commit_fetch_info_and_create_in_db_if_not_exists
    commit, created = _guts(ghcommit)
                      ^^^^^^^^^^^^^^^
  File "/app/conbench/entities/benchmark_result.py", line 1042, in _guts
    backfill_default_branch_commits(cinfo["repo_url"], dbcommit)
  File "/app/conbench/entities/commit.py", line 517, in backfill_default_branch_commits
    commits = _github.get_commits_to_branch(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/conbench/entities/commit.py", line 732, in get_commits_to_branch
    this_page = self._get_response(url + f"&page={page}")
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/conbench/entities/commit.py", line 860, in _get_response
    result = self._get_response_retry_guts(url)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/conbench/entities/commit.py", line 970, in _get_response_retry_guts
    raise Exception("Hourly GitHub HTTP API quota exhausted")
Exception: Hourly GitHub HTTP API quota exhausted

Traceback (most recent call last):
  File "/Users/dealeon/conbench_toy/client.py", line 19, in <module>
    adapter.post_results()
  File "/Users/dealeon/mambaforge/envs/conbench/lib/python3.12/site-packages/benchadapt/adapters/_adapter.py", line 174, in post_results
    res = client.post(path="/benchmarks/", json=result_dict)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dealeon/mambaforge/envs/conbench/lib/python3.12/site-packages/benchclients/base.py", line 64, in post
    self._maybe_raise(req_string=req_string, res=res)
  File "/Users/dealeon/mambaforge/envs/conbench/lib/python3.12/site-packages/benchclients/base.py", line 85, in _maybe_raise
    res.raise_for_status()
  File "/Users/dealeon/mambaforge/envs/conbench/lib/python3.12/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: INTERNAL SERVER ERROR for url: http://127.0.0.1:5000/api/benchmarks/

[austin3dickey]

Yes, this happens to me locally or in our CI sometimes as well. We've made efforts to mitigate this (see #917) but the underlying problem is that Conbench is fairly greedy about using the GitHub API. This issue would help a lot, but is a rather big project: #655

When you spin up a local Conbench server, do you provide it the GITHUB_API_TOKEN environment variable with your personal API key? If you aren't doing this, it's probably falling back to the unauthenticated method, which has significantly fewer calls per hour.

[DeaMariaLeon]

No, I was not providing my API key. Thank you.

[DeaMariaLeon]

If I do not provide my API key, it doesn't run at all. It is not falling to the unauthenticated method. Maybe this is normal, but just in case.