Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for sparse checkouts #1369

Merged
merged 4 commits into from Jun 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/test.yml
Expand Up @@ -72,6 +72,33 @@ jobs:
shell: bash
run: __test__/verify-side-by-side.sh

# Sparse checkout
- name: Sparse checkout
uses: ./
with:
sparse-checkout: |
__test__
.github
dist
path: sparse-checkout

- name: Verify sparse checkout
run: __test__/verify-sparse-checkout.sh

# Sparse checkout (non-cone mode)
- name: Sparse checkout (non-cone mode)
uses: ./
with:
sparse-checkout: |
/__test__/
/.github/
/dist/
sparse-checkout-cone-mode: false
path: sparse-checkout-non-cone-mode

- name: Verify sparse checkout (non-cone mode)
run: __test__/verify-sparse-checkout-non-cone-mode.sh

# LFS
- name: Checkout LFS
uses: ./
Expand Down
40 changes: 40 additions & 0 deletions README.md
Expand Up @@ -74,6 +74,15 @@ When Git 2.18 or higher is not in your PATH, falls back to the REST API to downl
# Default: true
clean: ''

# Do a sparse checkout on given patterns. Each pattern should be separated with
# new lines
# Default: null
sparse-checkout: ''

# Specifies whether to use cone-mode when doing a sparse checkout.
# Default: true
sparse-checkout-cone-mode: ''

# Number of commits to fetch. 0 indicates all history for all branches and tags.
# Default: 1
fetch-depth: ''
Expand Down Expand Up @@ -106,6 +115,9 @@ When Git 2.18 or higher is not in your PATH, falls back to the REST API to downl

# Scenarios

- [Fetch only the root files](#Fetch-only-the-root-files)
- [Fetch only the root files and `.github` and `src` folder](#Fetch-only-the-root-files-and-github-and-src-folder)
- [Fetch only a single file](#Fetch-only-a-single-file)
- [Fetch all history for all tags and branches](#Fetch-all-history-for-all-tags-and-branches)
- [Checkout a different branch](#Checkout-a-different-branch)
- [Checkout HEAD^](#Checkout-HEAD)
Expand All @@ -116,6 +128,34 @@ When Git 2.18 or higher is not in your PATH, falls back to the REST API to downl
- [Checkout pull request on closed event](#Checkout-pull-request-on-closed-event)
- [Push a commit using the built-in token](#Push-a-commit-using-the-built-in-token)

## Fetch only the root files

```yaml
- uses: actions/checkout@v3
with:
sparse-checkout: .
```

## Fetch only the root files and `.github` and `src` folder

```yaml
- uses: actions/checkout@v3
with:
sparse-checkout: |
.github
src
```

## Fetch only a single file
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you might provide another example for fetch single file that is not on the root dir? ex: few directory deep, etc.
also, maybe some examples of using glob pattern, ex checkout all *.md files? 😄

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's sounds great! 🚀

And maybe it has sense to add an example with lfs? In order to let know that the fetch of lfs can be reduced with the sparse-checkout option to 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you might provide another example for fetch single file that is not on the root dir? ex: few directory deep, etc. also, maybe some examples of using glob pattern, ex checkout all *.md files? 😄

That would require non-cone mode, which is deprecated by the Git project: https://git-scm.com/docs/git-sparse-checkout#_internalscone_pattern_set

Seeing as it is deprecated, I would like to avoid encouraging users to use it. Read: I do not want to add documentation how to do this, giving users the impression that they should use this.

And maybe it has sense to add an example with lfs? In order to let know that the fetch of lfs can be reduced with the sparse-checkout option to 🤔

The first part would be easy, by adding another LFS step that simply uses sparse-checkout: ..

However, the second part would be a bit tricky because this test requires a branch with LFS objects, and it uses the test-data/v2/lfs branch of actions/checkout itself. As you can see here, there is only one LFS file, in the root directory. Therefore, even in sparse checkout mode "all the LFS files" are fetched, i.e. that single one.

It would require adding another branch to the repository, with an additional LFS file in a subdirectory, and then we would still need to figure out a robust way to verify that it was not fetched, most likely relying on implementation details such as the location of the LFS cache (.git/lfs/objects) and the cache files' names, which would be fragile because implementation details are not guaranteed to remain unchanged.

So I don't know whether it is worth the effort.


```yaml
- uses: actions/checkout@v3
with:
sparse-checkout: |
README.md
sparse-checkout-cone-mode: false
```

## Fetch all history for all tags and branches

```yaml
Expand Down
4 changes: 4 additions & 0 deletions __test__/git-auth-helper.test.ts
Expand Up @@ -727,6 +727,8 @@ async function setup(testName: string): Promise<void> {
branchDelete: jest.fn(),
branchExists: jest.fn(),
branchList: jest.fn(),
sparseCheckout: jest.fn(),
sparseCheckoutNonConeMode: jest.fn(),
checkout: jest.fn(),
checkoutDetach: jest.fn(),
config: jest.fn(
Expand Down Expand Up @@ -800,6 +802,8 @@ async function setup(testName: string): Promise<void> {
authToken: 'some auth token',
clean: true,
commit: '',
sparseCheckout: [],
sparseCheckoutConeMode: true,
fetchDepth: 1,
lfs: false,
submodules: false,
Expand Down
14 changes: 12 additions & 2 deletions __test__/git-command-manager.test.ts
Expand Up @@ -39,7 +39,12 @@ describe('git-auth-helper tests', () => {
jest.spyOn(exec, 'exec').mockImplementation(mockExec)
const workingDirectory = 'test'
const lfs = false
git = await commandManager.createCommandManager(workingDirectory, lfs)
const doSparseCheckout = false
git = await commandManager.createCommandManager(
workingDirectory,
lfs,
doSparseCheckout
)

let branches = await git.branchList(false)

Expand Down Expand Up @@ -70,7 +75,12 @@ describe('git-auth-helper tests', () => {
jest.spyOn(exec, 'exec').mockImplementation(mockExec)
const workingDirectory = 'test'
const lfs = false
git = await commandManager.createCommandManager(workingDirectory, lfs)
const doSparseCheckout = false
git = await commandManager.createCommandManager(
workingDirectory,
lfs,
doSparseCheckout
)

let branches = await git.branchList(false)

Expand Down
2 changes: 2 additions & 0 deletions __test__/git-directory-helper.test.ts
Expand Up @@ -462,6 +462,8 @@ async function setup(testName: string): Promise<void> {
branchList: jest.fn(async () => {
return []
}),
sparseCheckout: jest.fn(),
sparseCheckoutNonConeMode: jest.fn(),
checkout: jest.fn(),
checkoutDetach: jest.fn(),
config: jest.fn(),
Expand Down
2 changes: 2 additions & 0 deletions __test__/input-helper.test.ts
Expand Up @@ -79,6 +79,8 @@ describe('input-helper tests', () => {
expect(settings.clean).toBe(true)
expect(settings.commit).toBeTruthy()
expect(settings.commit).toBe('1234567890123456789012345678901234567890')
expect(settings.sparseCheckout).toBe(undefined)
expect(settings.sparseCheckoutConeMode).toBe(true)
expect(settings.fetchDepth).toBe(1)
expect(settings.lfs).toBe(false)
expect(settings.ref).toBe('refs/heads/some-ref')
Expand Down
51 changes: 51 additions & 0 deletions __test__/verify-sparse-checkout-non-cone-mode.sh
@@ -0,0 +1,51 @@
#!/bin/sh

# Verify .git folder
if [ ! -d "./sparse-checkout-non-cone-mode/.git" ]; then
echo "Expected ./sparse-checkout-non-cone-mode/.git folder to exist"
exit 1
fi

# Verify sparse-checkout (non-cone-mode)
cd sparse-checkout-non-cone-mode

ENABLED=$(git config --local --get-all core.sparseCheckout)

if [ "$?" != "0" ]; then
echo "Failed to verify that sparse-checkout is enabled"
exit 1
fi

# Check that sparse-checkout is enabled
if [ "$ENABLED" != "true" ]; then
echo "Expected sparse-checkout to be enabled (is: $ENABLED)"
exit 1
fi

SPARSE_CHECKOUT_FILE=$(git rev-parse --git-path info/sparse-checkout)

if [ "$?" != "0" ]; then
echo "Failed to validate sparse-checkout"
exit 1
fi

# Check that sparse-checkout list is not empty
if [ ! -f "$SPARSE_CHECKOUT_FILE" ]; then
echo "Expected sparse-checkout file to exist"
exit 1
fi

# Check that all folders from sparse-checkout exists
for pattern in $(cat "$SPARSE_CHECKOUT_FILE")
do
if [ ! -d "${pattern#/}" ]; then
echo "Expected directory '${pattern#/}' to exist"
exit 1
fi
done

# Verify that the root directory is not checked out
if [ -f README.md ]; then
echo "Expected top-level files not to exist"
exit 1
fi
63 changes: 63 additions & 0 deletions __test__/verify-sparse-checkout.sh
@@ -0,0 +1,63 @@
#!/bin/sh

# Verify .git folder
if [ ! -d "./sparse-checkout/.git" ]; then
echo "Expected ./sparse-checkout/.git folder to exist"
exit 1
fi

# Verify sparse-checkout
cd sparse-checkout

SPARSE=$(git sparse-checkout list)

if [ "$?" != "0" ]; then
echo "Failed to validate sparse-checkout"
exit 1
fi

# Check that sparse-checkout list is not empty
if [ -z "$SPARSE" ]; then
echo "Expected sparse-checkout list to not be empty"
exit 1
fi

# Check that all folders of the sparse checkout exist
for pattern in $SPARSE
do
if [ ! -d "$pattern" ]; then
echo "Expected directory '$pattern' to exist"
exit 1
fi
done

checkSparse () {
if [ ! -d "./$1" ]; then
echo "Expected directory '$1' to exist"
exit 1
fi

for file in $(git ls-tree -r --name-only HEAD $1)
do
if [ ! -f "$file" ]; then
echo "Expected file '$file' to exist"
exit 1
fi
done
}

# Check that all folders and their children have been checked out
checkSparse __test__
checkSparse .github
checkSparse dist

# Check that only sparse-checkout folders have been checked out
for pattern in $(git ls-tree --name-only HEAD)
do
if [ -d "$pattern" ]; then
if [[ "$pattern" != "__test__" && "$pattern" != ".github" && "$pattern" != "dist" ]]; then
echo "Expected directory '$pattern' to not exist"
exit 1
fi
fi
done
9 changes: 9 additions & 0 deletions action.yml
Expand Up @@ -53,6 +53,15 @@ inputs:
clean:
description: 'Whether to execute `git clean -ffdx && git reset --hard HEAD` before fetching'
default: true
sparse-checkout:
description: >
Do a sparse checkout on given patterns.
Each pattern should be separated with new lines
default: null
sparse-checkout-cone-mode:
description: >
Specifies whether to use cone-mode when doing a sparse checkout.
default: true
fetch-depth:
description: 'Number of commits to fetch. 0 indicates all history for all branches and tags.'
default: 1
Expand Down