Add support for sparse checkouts

[dscho]

A relatively recent feature of Git are the sparse checkouts, i.e. checkouts where only part of the files in the revision are actually written to the worktree, and other files are only in the index. Combined with the partial clone feature, this is a very powerful tool to work e.g. with monorepos.

This PR adds support for sparse checkouts:

- uses: actions/checkout@v3
  with:
    sparse-checkout: |
      .github
      src

This PR builds on #1317, adding a commit to optionally turn off cone mode.

[dscho]

[Extracted from the PR description to avoid making it a part of the commit message when squash-merging:]

Apart from squashing commits to avoid introducing bugs only to immediately fix them, the fixups on top of #1317 include reverts of unrelated changes (there are some var declarations that should be const, but that fix has nothing to do with sparse checkouts and should therefore not be part of this PR).

Diff of the fixups on top of #1317

diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 590da62..8d5fffd 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -72,7 +72,7 @@ jobs:
         shell: bash
         run: __test__/verify-side-by-side.sh
 
-     # Sparse checkout
+      # Sparse checkout
       - name: Sparse checkout
         uses: ./
         with:
@@ -82,9 +82,7 @@ jobs:
             dist
           path: sparse-checkout
 
-      - name: Verify sparse checkout basic
-        run: __test__/verify-sparse-checkout-basic.sh
-      - name: Verify sparse checkout example
+      - name: Verify sparse checkout
         run: __test__/verify-sparse-checkout.sh
 
       # LFS
@@ -220,7 +218,7 @@ jobs:
           path: basic
       - name: Verify basic
         run: __test__/verify-basic.sh --archive
-
+    
   test-git-container:
     runs-on: ubuntu-latest
     container: bitnami/git:latest
diff --git a/README.md b/README.md
index f46b30a..14b4366 100644
--- a/README.md
+++ b/README.md
@@ -74,7 +74,7 @@ When Git 2.18 or higher is not in your PATH, falls back to the REST API to downl
     # Default: true
     clean: ''
 
-    # Do a sparse checkout on given patterns. Each pattern should be sepparated with
+    # Do a sparse checkout on given patterns. Each pattern should be separated with
     # new lines
     # Default: null
     sparse-checkout: ''
@@ -111,17 +111,17 @@ When Git 2.18 or higher is not in your PATH, falls back to the REST API to downl
 
 # Scenarios
 
-- [Fetch only the root files](#fetch-only-the-root-files)
-- [Fetch only the root files and `.github` and `src` folder](#fetch-only-the-root-files-and-github-and-src-folder)
-- [Fetch all history for all tags and branches](#fetch-all-history-for-all-tags-and-branches)
-- [Checkout a different branch](#checkout-a-different-branch)
-- [Checkout HEAD^](#checkout-head)
-- [Checkout multiple repos (side by side)](#checkout-multiple-repos-side-by-side)
-- [Checkout multiple repos (nested)](#checkout-multiple-repos-nested)
-- [Checkout multiple repos (private)](#checkout-multiple-repos-private)
-- [Checkout pull request HEAD commit instead of merge commit](#checkout-pull-request-head-commit-instead-of-merge-commit)
-- [Checkout pull request on closed event](#checkout-pull-request-on-closed-event)
-- [Push a commit using the built-in token](#push-a-commit-using-the-built-in-token)
+- [Fetch only the root files](#Fetch-only-the-root-files)
+- [Fetch only the root files and `.github` and `src` folder](#Fetch-only-the-root-files-and-github-and-src-folder)
+- [Fetch all history for all tags and branches](#Fetch-all-history-for-all-tags-and-branches)
+- [Checkout a different branch](#Checkout-a-different-branch)
+- [Checkout HEAD^](#Checkout-HEAD)
+- [Checkout multiple repos (side by side)](#Checkout-multiple-repos-side-by-side)
+- [Checkout multiple repos (nested)](#Checkout-multiple-repos-nested)
+- [Checkout multiple repos (private)](#Checkout-multiple-repos-private)
+- [Checkout pull request HEAD commit instead of merge commit](#Checkout-pull-request-HEAD-commit-instead-of-merge-commit)
+- [Checkout pull request on closed event](#Checkout-pull-request-on-closed-event)
+- [Push a commit using the built-in token](#Push-a-commit-using-the-built-in-token)
 
 ## Fetch only the root files
 
diff --git a/__test__/verify-sparse-checkout-basic.sh b/__test__/verify-sparse-checkout-basic.sh
deleted file mode 100755
index f8085f4..0000000
--- a/__test__/verify-sparse-checkout-basic.sh
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/bin/bash
-
-# Verify .git folder
-if [ ! -d "./sparse-checkout/.git" ]; then
-  echo "Expected ./sparse-checkout/.git folder to exist"
-  exit 1
-fi
-
-# Verify sparse-checkout basic
-cd sparse-checkout
-
-SPARSE=$(git sparse-checkout list)
-
-if [ "$?" != "0" ]; then
-    echo "Failed to validate sparse-checkout"
-    exit 1
-fi
-
-# Check that sparse-checkout list is not empty
-if [ -z "$SPARSE" ]; then
-  echo "Expected sparse-checkout list to not be empty"
-  exit 1
-fi
-
-# Check that all folders from sparse-checkout exists
-for pattern in $(git sparse-checkout list)
-do
-  if [ ! -d "$pattern" ]; then
-    echo "Expected directory '$pattern' to exist"
-    exit 1
-  fi
-done
\ No newline at end of file
diff --git a/__test__/verify-sparse-checkout.sh b/__test__/verify-sparse-checkout.sh
index b0dd060..f668430 100755
--- a/__test__/verify-sparse-checkout.sh
+++ b/__test__/verify-sparse-checkout.sh
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/sh
 
 # Verify .git folder
 if [ ! -d "./sparse-checkout/.git" ]; then
@@ -9,6 +9,28 @@ fi
 # Verify sparse-checkout
 cd sparse-checkout
 
+SPARSE=$(git sparse-checkout list)
+
+if [ "$?" != "0" ]; then
+    echo "Failed to validate sparse-checkout"
+    exit 1
+fi
+
+# Check that sparse-checkout list is not empty
+if [ -z "$SPARSE" ]; then
+  echo "Expected sparse-checkout list to not be empty"
+  exit 1
+fi
+
+# Check that all folders of the sparse checkout exist
+for pattern in $SPARSE
+do
+  if [ ! -d "$pattern" ]; then
+    echo "Expected directory '$pattern' to exist"
+    exit 1
+  fi
+done
+
 checkSparse () {
   if [ ! -d "./$1" ]; then
     echo "Expected directory '$1' to exist"
@@ -24,12 +46,12 @@ checkSparse () {
   done
 }
 
-# Check that all folders and its childrens has been fetched correctly
+# Check that all folders and their children have been checked out
 checkSparse __test__
 checkSparse .github
 checkSparse dist
 
-# Check that only sparse-checkout folders has been fetched
+# Check that only sparse-checkout folders have been checked out
 for pattern in $(git ls-tree --name-only HEAD)
 do
   if [ -d "$pattern" ]; then
diff --git a/action.yml b/action.yml
index bb7451e..d0c96b1 100644
--- a/action.yml
+++ b/action.yml
@@ -56,7 +56,7 @@ inputs:
   sparse-checkout:
     description: >
       Do a sparse checkout on given patterns.
-      Each pattern should be sepparated with new lines
+      Each pattern should be separated with new lines
     default: null
   fetch-depth:
     description: 'Number of commits to fetch. 0 indicates all history for all branches and tags.'
diff --git a/dist/index.js b/dist/index.js
index c7aee5b..b48bffb 100644
--- a/dist/index.js
+++ b/dist/index.js
@@ -704,8 +704,8 @@ class GitCommandManager {
     }
     log1(format) {
         return __awaiter(this, void 0, void 0, function* () {
-            const args = format ? ['log', '-1', format] : ['log', '-1'];
-            const silent = format ? false : true;
+            var args = format ? ['log', '-1', format] : ['log', '-1'];
+            var silent = format ? false : true;
             const output = yield this.execGit(args, false, silent);
             return output.stdout;
         });
diff --git a/src/git-command-manager.ts b/src/git-command-manager.ts
index 132f34e..3a0ec58 100644
--- a/src/git-command-manager.ts
+++ b/src/git-command-manager.ts
@@ -308,8 +308,8 @@ class GitCommandManager {
   }
 
   async log1(format?: string): Promise<string> {
-    const args = format ? ['log', '-1', format] : ['log', '-1']
-    const silent = format ? false : true
+    var args = format ? ['log', '-1', format] : ['log', '-1']
+    var silent = format ? false : true
     const output = await this.execGit(args, false, silent)
     return output.stdout
   }

[dfdez]

Hey @dscho I appreciate your participation on this topic!

But why did you create a new PR instead of continue the work in the original one?

Mainly to avoid more PRs to discuss and make easier go forward with this topic

I was working on the comments of #1317 (comment) and creating a new PR will only create conflicts and more difficulties to achieve this goal

Thanks anyway and I will take a look to the diff you proposed!

[TingluoHuang]

Suggested change

	result.sparseCheckoutConeMode =
	(core.getInput('sparse-checkout-cone-mode') || 'true').toUpperCase() !==
	'FALSE'
	result.sparseCheckoutConeMode =
	(core.getInput('sparse-checkout-cone-mode') || 'true').toUpperCase() === 'TRUE'

I think we normally compare with TRUE in the input-helper.ts

[dscho]

I fixed it in my most recent force-push.

[TingluoHuang]

do we need to have extra check to make sure git is greater than certain version that support sparse-checkout?

[dfdez]

Hey @TingluoHuang git sparse-checkout is supported since the git version 2.26.0 and the current version on actions/checkout of git is 2.40.1 so I don't think this is something necessary 🤔

[dscho]

The current Git version on hosted runners is v2.40.1. But actions/checkout also supports self-hosted runners where users are often not at liberty to bump the Git version, e.g. because they're stuck on old Ubuntu LTS versions which simply refuse to move the needle.

I added c4602b4 to address this.

[dfdez]

Uu nice! I understand perfect ✨

[dfdez]

Hey @dscho after taking a look to this PR there are interesting changes that you made like some misspelling words, corrections to the README.md and the unification of the sparse-checkout test in only one file that I see perfect!

That is why I have made those updates on my branch on this commits d8614bd 7301241

Regarding the non-cone mode as mentioned on the git documentation we shouldn't use it.

Also is true that the var change to const does nothing to do with the goal of my PR but I thought it wouldn't be a problem to clean that!

Thanks for all the work done and let's hope we can use sparse-checkout soon!

[dscho]

But why did you create a new PR instead of continue the work in the original one?

I hoped that I could help move the feature along faster that way.

Regarding the non-cone mode as mentioned on the git documentation we shouldn't use it.

It is true that it is deprecated. And for good reasons: the performance of non-cone mode gets really bad really quickly with increasing sparse-checkout definitions.

However, it is precisely this mode that I require in a couple of legacy projects where code mode is just not feasible.

[dfdez]

I hoped that I could help move the feature along faster that way.

Yes you are right but maybe it could be a better idea to create that PR to mine instead of creating a new one? But it's fine! Also I see that you are a contributor therefore I guess your PR have more priority?

However, it is precisely this mode that I require in a couple of legacy projects where code mode is just not feasible.

Okay that is a completely valid reason, but having this functionality knowing that is deprecated I wasn't sure if was the right thing to do, but if it's not a problem I don't see why we shouldn't add it

Also I have an answer regarding this comment about lfs and sparse-checkout let me know you opinion on this!

I'm not sure how we should proceed now, do I continue the changes on my current PR merging yours? Or do we continue on this PR?

[dscho]

you are a contributor therefore I guess your PR have more priority?

That was my thinking.

Also I have an answer regarding this comment about lfs and sparse-checkout let me know you opinion on this!

I don't really have any opinion on this, I am not using Git LFS in my repositories. If you have a solution that is working for everyone involved, I'd be happy to add it to this here PR (in the hopes that it will get merged soon).

[dfdez]

I'm making some test to see that everything works as expected I will let you know if a see any strange behaviour 👍🏽

[dfdez]

All seems to be good! 🚀

• Sparse checkout with cone
• Sparse checkout with non-cone mode
• Sparse checkout with cone and lfs
• Sparse checkout with non-cone mode and lfs

[TingluoHuang]

do you might provide another example for fetch single file that is not on the root dir? ex: few directory deep, etc.
also, maybe some examples of using glob pattern, ex checkout all *.md files? 😄

[dfdez]

That's sounds great! 🚀

And maybe it has sense to add an example with lfs? In order to let know that the fetch of lfs can be reduced with the sparse-checkout option to 🤔

[dscho]

do you might provide another example for fetch single file that is not on the root dir? ex: few directory deep, etc. also, maybe some examples of using glob pattern, ex checkout all *.md files? 😄

That would require non-cone mode, which is deprecated by the Git project: https://git-scm.com/docs/git-sparse-checkout#_internalscone_pattern_set

Seeing as it is deprecated, I would like to avoid encouraging users to use it. Read: I do not want to add documentation how to do this, giving users the impression that they should use this.

And maybe it has sense to add an example with lfs? In order to let know that the fetch of lfs can be reduced with the sparse-checkout option to 🤔

The first part would be easy, by adding another LFS step that simply uses sparse-checkout: ..

However, the second part would be a bit tricky because this test requires a branch with LFS objects, and it uses the test-data/v2/lfs branch of actions/checkout itself. As you can see here, there is only one LFS file, in the root directory. Therefore, even in sparse checkout mode "all the LFS files" are fetched, i.e. that single one.

It would require adding another branch to the repository, with an additional LFS file in a subdirectory, and then we would still need to figure out a robust way to verify that it was not fetched, most likely relying on implementation details such as the location of the LFS cache (.git/lfs/objects) and the cache files' names, which would be fragile because implementation details are not guaranteed to remain unchanged.

So I don't know whether it is worth the effort.

[TingluoHuang]

what will happen if sparseCheckout is an empty array?

[dfdez]

That shouldn't happen, you can see that the length is checked when setting up the sparseCheckout variable

You can find this here

[Spiritaine]

Good review

[Spiritaine]

Good review