Update actions/checkout action to v3 #9527

HebaruSan · 2023-01-17T22:59:43Z

Problem

The metadata tester is broken. It now emits this every time it runs:

  Traceback (most recent call last):
    File "/usr/local/bin/ckanmetatester", line 11, in <module>
      load_entry_point('CkanMetaTester==0.1', 'console_scripts', 'ckanmetatester')()
    File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/__init__.py", line 24, in test_metadata
      environ.get('INPUT_DIFF_META_ROOT'))
    File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/ckan_meta_tester.py", line 74, in test_metadata
      for file in self.files_to_test(source):
    File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/ckan_meta_tester.py", line 231, in files_to_test
      return self.paths_from_diff(self.branch_diff(Repo('.')))
    File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/ckan_meta_tester.py", line 241, in branch_diff
      logging.debug('Looking for merge base between %s and %s', start_ref, repo.head.commit.hexsha)
    File "/usr/local/lib/python3.7/dist-packages/git/refs/symbolic.py", line 226, in _get_commit
      obj = self._get_object()
    File "/usr/local/lib/python3.7/dist-packages/git/refs/symbolic.py", line 219, in _get_object
      return Object.new_from_sha(self.repo, hex_to_bin(self.dereference_recursive(self.repo, self.path)))
    File "/usr/local/lib/python3.7/dist-packages/git/objects/base.py", line 94, in new_from_sha
      oinfo = repo.odb.info(sha1)
    File "/usr/local/lib/python3.7/dist-packages/git/db.py", line 40, in info
      hexsha, typename, size = self._git.get_object_header(bin_to_hex(binsha))
    File "/usr/local/lib/python3.7/dist-packages/git/cmd.py", line [13](https://github.com/KSP-CKAN/NetKAN/actions/runs/3895070850/jobs/6678636081#step:6:14)83, in get_object_header
      return self.__get_object_header(cmd, ref)
    File "/usr/local/lib/python3.7/dist-packages/git/cmd.py", line 1370, in __get_object_header
      return self._parse_object_header(cmd.stdout.readline())
    File "/usr/local/lib/python3.7/dist-packages/git/cmd.py", line 13[29](https://github.com/KSP-CKAN/NetKAN/actions/runs/3895070850/jobs/6678636081#step:6:30), in _parse_object_header
      raise ValueError("SHA could not be resolved, git returned: %r" % (header_line.strip()))
  ValueError: SHA could not be resolved, git returned: b''

Causes (speculative)

git added a "safe directories" feature in mid-2022, which causes it to print an error without even parsing the repo config if the current user doesn't own a repo, to prevent other users from forcing the current user to run code. Adding KSP 1.12.5 to the embedded build map probably updated the docker image to use a newer version of git including these changes.
Presumably such an ownership discrepancy exists somewhere in the labyrinthine gnarled bowels of the GitHub workflow runtime
GitPython doesn't handle this error intelligently, see Cannot retrieve commits from a specific repository gitpython-developers/GitPython#1016; it just returns an empty string somewhere internally, confusing itself

I have not personally loaded up the current docker image and simulated running it in order to probe the details of exactly what's happening in the filesystem. But I find this a plausible explanation for the problem, and if it doesn't work out, we can continue investigating and try something else.

Changes

Now we upgrade from v2 to v3 of actions/checkout, which ignores "safe directories" by default (see actions/checkout#762 and actions/checkout#770), and should mitigate this if that's the cause.

See KSP-CKAN/xKAN-meta_testing#89, which this doesn't seem to actually fix.

HebaruSan · 2023-01-17T23:01:35Z

Nope, still happens. Delightful.

Ahh, the /usr/bin/git config --global --add safe.directory /home/runner/work/NetKAN/NetKAN command was already present in the v2 runs. Hmm.

HebaruSan · 2023-01-18T17:08:16Z

git --version output: git version 2.39.0
git rev-parse HEAD output: b74473bae858f23592a92c6403b46a463007de30

HebaruSan · 2023-01-18T17:12:24Z

But the git version and global config can be different inside the docker image, right?

HebaruSan · 2023-01-18T17:17:34Z

We can try adding git config --global --add safe.directory '*' to the Dockerfile, but that would require committing to the CKAN repo's master branch, and I'm not sure how I feel about doing that for a mere guess...

HebaruSan · 2023-01-18T17:31:14Z

$ docker run -it --entrypoint bash kspckan/metadata
root@51cf9f68ad0d:/# git --version
git version 2.20.1

I think that version is from 2018, so it probably doesn't have the safe directory feature.

... but I don't have a repo created by actions/checkout for testing.

HebaruSan · 2023-01-18T18:07:15Z

Finally got the error to happen locally in my own NetKAN repo! The GitHub workflow prints the entire docker command that it runs, which is very helpful:

$ docker run --workdir /github/workspace -v $(pwd):/github/workspace -e INPUT_SOURCE=commits kspckan/metadata
Traceback (most recent call last):
  File "/usr/local/bin/ckanmetatester", line 11, in <module>
    load_entry_point('CkanMetaTester==0.1', 'console_scripts', 'ckanmetatester')()
  File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/__init__.py", line 24, in test_metadata
    environ.get('INPUT_DIFF_META_ROOT'))
  File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/ckan_meta_tester.py", line 74, in test_metadata
    for file in self.files_to_test(source):
  File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/ckan_meta_tester.py", line 231, in files_to_test
    return self.paths_from_diff(self.branch_diff(Repo('.')))
  File "/usr/local/lib/python3.7/dist-packages/ckan_meta_tester/ckan_meta_tester.py", line 241, in branch_diff
    logging.debug('Looking for merge base between %s and %s', start_ref, repo.head.commit.hexsha)
  File "/usr/local/lib/python3.7/dist-packages/git/refs/symbolic.py", line 226, in _get_commit
    obj = self._get_object()
  File "/usr/local/lib/python3.7/dist-packages/git/refs/symbolic.py", line 219, in _get_object
    return Object.new_from_sha(self.repo, hex_to_bin(self.dereference_recursive(self.repo, self.path)))
  File "/usr/local/lib/python3.7/dist-packages/git/objects/base.py", line 94, in new_from_sha
    oinfo = repo.odb.info(sha1)
  File "/usr/local/lib/python3.7/dist-packages/git/db.py", line 40, in info
    hexsha, typename, size = self._git.get_object_header(bin_to_hex(binsha))
  File "/usr/local/lib/python3.7/dist-packages/git/cmd.py", line 1383, in get_object_header
    return self.__get_object_header(cmd, ref)
  File "/usr/local/lib/python3.7/dist-packages/git/cmd.py", line 1370, in __get_object_header
    return self._parse_object_header(cmd.stdout.readline())
  File "/usr/local/lib/python3.7/dist-packages/git/cmd.py", line 1329, in _parse_object_header
    raise ValueError("SHA could not be resolved, git returned: %r" % (header_line.strip()))
ValueError: SHA could not be resolved, git returned: b''

HebaruSan · 2023-01-18T18:09:36Z

$ docker run -it --workdir /github/workspace -v $(pwd):/github/workspace -e INPUT_SOURCE=commits --entrypoint bash kspckan/metadata
root@937525a4ec61:/github/workspace# git rev-parse HEAD
fatal: detected dubious ownership in repository at '/github/workspace'
To add an exception for this directory, call:

        git config --global --add safe.directory /github/workspace

!!!!!

So I guess the safe directory feature is a little older than mid-2022; at least back to 2018!

HebaruSan · 2023-02-08T07:49:43Z

Your branch is really old. Try rebasing on current master.

HebaruSan · 2023-02-11T21:14:46Z

Validation succeeded after the changes in KSP-CKAN/xKAN-meta_testing#90. 🎉
This one isn't needed anymore.

Update actions/checkout action to v3

286a9fb

HebaruSan added Pull request Bug labels Jan 17, 2023

HebaruSan added 3 commits January 17, 2023 23:21

Explicitly set git safe.directory

a9e4704

Run git rev-parse HEAD for testing

b74473b

Get git version

a860fd9

HebaruSan added the In progress label Jan 18, 2023

HebaruSan mentioned this pull request Jan 18, 2023

Set git safe.directory in meta tester KSP-CKAN/CKAN#3766

Merged

HebaruSan and others added 3 commits January 18, 2023 20:16

Remove change to trigger clean rebuild

600ad6c

Turn on debugging output

47cb4aa

ci: Test Debug Workflow

8e281e5

HebaruSan closed this Feb 11, 2023

HebaruSan deleted the fix/inflate-workflow branch February 11, 2023 21:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update actions/checkout action to v3 #9527

Update actions/checkout action to v3 #9527

HebaruSan commented Jan 17, 2023 •

edited

HebaruSan commented Jan 17, 2023 •

edited

HebaruSan commented Jan 18, 2023

HebaruSan commented Jan 18, 2023

HebaruSan commented Jan 18, 2023

HebaruSan commented Jan 18, 2023 •

edited

HebaruSan commented Jan 18, 2023

HebaruSan commented Jan 18, 2023 •

edited

HebaruSan commented Feb 8, 2023

HebaruSan commented Feb 11, 2023

Update actions/checkout action to v3 #9527

Update actions/checkout action to v3 #9527

Conversation

HebaruSan commented Jan 17, 2023 • edited

Problem

Causes (speculative)

Changes

HebaruSan commented Jan 17, 2023 • edited

HebaruSan commented Jan 18, 2023

HebaruSan commented Jan 18, 2023

HebaruSan commented Jan 18, 2023

HebaruSan commented Jan 18, 2023 • edited

HebaruSan commented Jan 18, 2023

HebaruSan commented Jan 18, 2023 • edited

HebaruSan commented Feb 8, 2023

HebaruSan commented Feb 11, 2023

HebaruSan commented Jan 17, 2023 •

edited

HebaruSan commented Jan 17, 2023 •

edited

HebaruSan commented Jan 18, 2023 •

edited

HebaruSan commented Jan 18, 2023 •

edited