Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Newly added sparse checkout functionality is not cleaning up after itself #1475

Closed
markjm opened this issue Sep 15, 2023 · 10 comments · Fixed by #1598
Closed

Newly added sparse checkout functionality is not cleaning up after itself #1475

markjm opened this issue Sep 15, 2023 · 10 comments · Fixed by #1598

Comments

@markjm
Copy link

markjm commented Sep 15, 2023

Thanks @dscho for contributing #1369 . We have started using it and noticed the following issue

repo-a / .github/workflows/workflow-a.yml

jobs:
  job-1:
    runs-on: internal-runner

- uses: actions/checkout@v3
  with:
    sparse-checkout: |
      .github
      src

running just the above works as expected. But consider another workflow in the same repo

repo-a / .github/workflows/workflow-b.yml

jobs:
  job-12:
    runs-on: internal-runner

- uses: actions/checkout@v3
- run: ls ./not-src

When running workflow-a, then workflow workflow-b, workflow-b will not find any files in ./not-src.

This is because workflow workflow-b runs in the same _work directory as workflow-a. On the first run of workflow-a, we have polluted the repo in a way this action doesn't know about (and thus can't clean up).

Notice that, even in workflow-b:

git config --list will show core.sparseCheckout true
git sparse-checkout list will still include the listed paths
and a simple
ls will only show the sparse directories for workflow-a

It seems that - similar to the auth cleanup (and other pre-run steps) this action does - we need to disable sparse checkout in the pre steps of this action

Note: this is likely only an issue for self-hosted runners which dont get the same cleaning/re-init process github-hosted runners get

Similar-ish issue to recently reported 47fbe2d (for submodules, instead of sparse checkout). @megamanics

@dscho
Copy link
Contributor

dscho commented Sep 18, 2023

this is likely only an issue for self-hosted runners

Indeed.

The working directory's path typically contains the workflow name, e.g. D:\a\_work\repo-a\repo-a, but not the workflow name.

A solid work-around would be to scorch the worktree (if it exists), right before the actions/checkout step.

A more surgical work-around would be to use a shell script like this:

- shell: bash
  run: |
    if test true = "$(git config --type bool core.sparsecheckout)"
    then
      rm "$(git rev-parse --git-path info/sparse-checkout)" &&
      git sparse-checkout disable
    fi

@markjm
Copy link
Author

markjm commented Sep 19, 2023

We are looking to use such a workaround in the interim, but it seems this is something that should be fixed here within actions/checkout, no?

@jinhyoo-mp
Copy link

This caused CI issues for us so we had to remove sparse checkout

@vibro
Copy link

vibro commented Jan 17, 2024

I also ran into issues with this, two different workflows, one which uses sparse and one which doesn't. I ended up removing the contents of the directory prior to checkout to force a clean checkout every time.

@aamagda
Copy link

aamagda commented Jan 29, 2024

@dscho Hi! Do you have plans to fix this isuue?

@dscho
Copy link
Contributor

dscho commented Jan 29, 2024

@aamagda time constraints force me to say "no".

Having said that, if you can start implementing that (probably by specifically resetting the sparse checkout if none is desired), and verify that it works, I can take it from there.

dscho added a commit to dscho/checkout that referenced this issue Jan 31, 2024
This should allow users to reuse existing folders when running
`actions/checkout` where a previous run asked for a sparse checkout but
the current run does not ask for a sparse checkout.

This fixes actions#1475

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
jww3 added a commit that referenced this issue Feb 21, 2024
When a worktree is reused by actions/checkout and the first time sparse checkout was enabled, we need to ensure that the second time it is only a sparse checkout if explicitly asked for. Otherwise, we need to disable the sparse checkout so that a full checkout is the outcome of this Action.

## Details
* If no `sparse-checkout` parameter is specified, disable it

This should allow users to reuse existing folders when running
`actions/checkout` where a previous run asked for a sparse checkout but
the current run does not ask for a sparse checkout.

This fixes #1475

There are use cases in particular with non-ephemeral (self-hosted) runners where an
existing worktree (that has been initialized as a sparse checkout) is
reused in subsequent CI runs (where `actions/checkout` is run _without_
any `sparse-checkout` parameter).

In these scenarios, we need to make sure that the sparse checkout is
disabled before checking out the files.

### Also includes:

* npm run build
* ci: verify that an existing sparse checkout can be made unsparse
* Added a clarifying comment about test branches.
* `test-proxy` now uses newly-minted `test-ubuntu-git` container image from ghcr.io

---------

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Co-authored-by: John Wesley Walker III <81404201+jww3@users.noreply.github.com>
@justin-newman
Copy link

Assuming v4.1.2 works when explicitly setting it, I am still having the same issue with v4.1.2 on a self-hosted server

@dscho
Copy link
Contributor

dscho commented Apr 8, 2024

Assuming v4.1.2 works when explicitly setting it, I am still having the same issue with v4.1.2 on a self-hosted server

@justin-newman could you verify in the logs that the git sparse-checkout disable command was run, like so (line 51)?

@justin-newman
Copy link

Assuming v4.1.2 works when explicitly setting it, I am still having the same issue with v4.1.2 on a self-hosted server

@justin-newman could you verify in the logs that the git sparse-checkout disable command was run, like so (line 51)?

Yes, it is doing that

@dscho
Copy link
Contributor

dscho commented Apr 12, 2024

could you verify in the logs that the git sparse-checkout disable command was run, like so (line 51)?

Yes, it is doing that

@justin-newman can you find out whether that command simply does not work in your setup? I am somewhat surprised to read that it was called yet did not fix the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants