Deleted files restored after committing with no files matching lint-staged config

[benediktvaldez]

Description

I have several edits right now and a few files that were deleted. I'm now in the process of committing my changes in smaller chunks for a sensible git history, and I noticed the files that had been deleted kept appearing in my repo again (not for every commit, just some). Turns out, when I don't stage the deleted files and none of the files being committed match the lint-staged config, they seem to reappear for some reason. If I create a new file or modify an existing file that matches, everything runs as expected. Note that this seems to apply to all deleted files, unrelated to whether they match the lint-staged config.

I suspect this might be related to #75 and/or #387 (#75 is a feature that has brought so much joy to my heart ❤️ - thank you for all your great work!)

Update: I tried changing my lint-staged config to target all files from root, including package.json, assuming it would fix the issue, but it prevailed. I added an additional debug log below. It is almost identical, it seems to process the package.json file this time as expected, but the error at the end seems to be the same.

Relevant parts of my package.json

{
  "scripts": {
    "format:pre-commit": "prettier --write",
    "lint:pre-commit": "tslint --config tslint.json --project tsconfig.json --fix --format codeFrame",
  },
  "lint-staged": {
    "src/**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx}": [
      "yarn format:pre-commit",
      "git add"
    ],
    "src/**/*.{ts,tsx}": [
      "yarn lint:pre-commit",
      "git add"
    ]
  },
  "husky": {
    "hooks": {
      "pre-commit": "lint-staged --debug"
    }
  }
}

On the second run I replaced the src in lint-staged with . resulting in this

{
  "lint-staged": {
    "./**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx}": [
      "yarn format:pre-commit",
      "git add"
    ],
    "./**/*.{ts,tsx}": [
      "yarn lint:pre-commit",
      "git add"
    ]
  },
}

Steps to reproduce

• Delete some file
• Edit a file that won't match your lint-staged config (I only target my src file, so I edited my package.json as an example)
• Stage the edited file while leaving the deleted file unstaged. My git status looked like this

➜ git status
On branch develop
Changes to be committed:
	modified:   package.json

Changes not staged for commit:
	deleted:    .eslintrc
	deleted:    src/styles/Breakpoints.ts
	deleted:    src/styles/Theme.ts

• Notice how the deleted file reappears after the pre-commit hook runs (and returns a success, since no files needed to be touched). git status claims to have a clean working tree, but the files should still be deleted.

➜ git status
On branch develop
nothing to commit, working tree clean

Debug Logs

expand to view debug log when only targeting `./src`

➜ git commit -m 'trigger lint-staged debug log'
husky > pre-commit (node v11.4.0)
  lint-staged:bin Running `lint-staged@8.1.0` +0ms
  lint-staged:find-bin Loaded package.json using `process.cwd()` +0ms
  lint-staged Loading config using `cosmiconfig` +0ms
  lint-staged Successfully loaded config from `/path/to/project/package.json`:
  lint-staged { 'src/**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx}': [ 'yarn format:pre-commit', 'git add' ],
  lint-staged   'src/**/*.{ts,tsx}': [ 'yarn lint:pre-commit', 'git add' ] } +6ms
  lint-staged:cfg Normalizing config +0ms
  lint-staged:cfg Validating config +2ms
Running lint-staged with the following config:
{
  linters: {
    'src/**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx}': [
      'yarn format:pre-commit',
      'git add'
    ],
    'src/**/*.{ts,tsx}': [
      'yarn lint:pre-commit',
      'git add'
    ]
  },
  concurrent: true,
  chunkSize: 9007199254740991,
  globOptions: {
    matchBase: true,
    dot: true
  },
  ignore: [],
  subTaskConcurrency: 1,
  renderer: 'verbose',
  relative: false
}
  lint-staged:run Running all linter scripts +0ms
  lint-staged:run Resolved git directory to be `/path/to/project` +0ms
  lint-staged:run Loaded list of staged files in git:
  lint-staged:run [ 'package.json' ] +27ms
  lint-staged:gen-tasks Generating linter tasks +0ms
  lint-staged:cfg Normalizing config +33ms
  lint-staged:gen-tasks Generated task:
  lint-staged:gen-tasks { pattern: 'src/**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx}',
  lint-staged:gen-tasks   commands: [ 'yarn format:pre-commit', 'git add' ],
  lint-staged:gen-tasks   fileList: [] } +17ms
  lint-staged:gen-tasks Generated task:
  lint-staged:gen-tasks { pattern: 'src/**/*.{ts,tsx}',
  lint-staged:gen-tasks   commands: [ 'yarn lint:pre-commit', 'git add' ],
  lint-staged:gen-tasks   fileList: [] } +2ms
Stashing changes... [started]
  lint-staged:git Stashing files... +0ms
  lint-staged:git Running git command [ 'write-tree' ] +0ms
  lint-staged:git Running git command [ 'add', '.' ] +19ms
  lint-staged:git Running git command [ 'write-tree' ] +28ms
  lint-staged:git Running git command [ 'read-tree', '084a2c1bba523fd9fa8f8a34c09170262da019f0' ] +20ms
  lint-staged:git Running git command [ 'checkout-index', '-af' ] +24ms
  lint-staged:git Done stashing files! +48ms
Stashing changes... [completed]
Running linters... [started]
Running tasks for src/**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx} [started]
Running tasks for src/**/*.{ts,tsx} [started]
Running tasks for src/**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx} [skipped]
→ No staged files match src/**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx}
Running tasks for src/**/*.{ts,tsx} [skipped]
→ No staged files match src/**/*.{ts,tsx}
Running linters... [completed]
Updating stash... [started]
  lint-staged:git Running git command [ 'write-tree' ] +2ms
Updating stash... [completed]
Restoring local changes... [started]
  lint-staged:git Restoring working copy +15ms
  lint-staged:git Running git command [ 'read-tree', 'd57d961d2cf7f14ce74923f15e781f6d1502c1a2' ] +0ms
  lint-staged:git Running git command [ 'checkout-index', '-af' ] +23ms
  lint-staged:git Restoring index with formatting changes +50ms
  lint-staged:git Running git command [ 'read-tree', '084a2c1bba523fd9fa8f8a34c09170262da019f0' ] +0ms
  lint-staged:git Generating diff between trees 084a2c1bba523fd9fa8f8a34c09170262da019f0 and 084a2c1bba523fd9fa8f8a34c09170262da019f0... +17ms
  lint-staged:git Running git command [ 'diff-tree',
  '--ignore-submodules',
  '--binary',
  '--no-color',
  '--no-ext-diff',
  '--unified=0',
  '084a2c1bba523fd9fa8f8a34c09170262da019f0',
  '084a2c1bba523fd9fa8f8a34c09170262da019f0' ] +0ms
  lint-staged:git Running git command [ 'apply',
  '-v',
  '--whitespace=nowarn',
  '--reject',
  '--recount',
  '--unidiff-zero' ] +16ms
  lint-staged:git Could not apply patch to the stashed files cleanly +18ms
  lint-staged:git Error: Error: Command failed: git apply -v --whitespace=nowarn --reject --recount --unidiff-zero
  lint-staged:git error: unrecognized input
  lint-staged:git
  lint-staged:git
  lint-staged:git     at execGit (/path/to/project/node_modules/lint-staged/src/gitWorkflow.js:28:11)
  lint-staged:git     at process.internalTickCallback (internal/process/next_tick.js:77:7) +0ms
  lint-staged:git Patch content: +0ms
  lint-staged:git
  lint-staged:git  +1ms
  lint-staged:git Found conflicts between formatters and local changes. Formatters changes will be ignored for conflicted hunks. +0ms
  lint-staged:git Deleted files and folders:
  lint-staged:git   +5ms
Restoring local changes... [completed]
  lint-staged linters were executed successfully! +381ms
[develop 505cd68] trigger lint-staged debug log
 1 file changed, 1 insertion(+), 1 deletion(-)

expand to view debug log when targeting all files in `./`

➜ git commit -m 'trigger lint-staged debug log'
husky > pre-commit (node v11.4.0)
  lint-staged:bin Running `lint-staged@8.1.0` +0ms
  lint-staged:find-bin Loaded package.json using `process.cwd()` +0ms
  lint-staged Loading config using `cosmiconfig` +0ms
  lint-staged Successfully loaded config from `/path/to/project/package.json`:
  lint-staged { './**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx}': [ 'yarn format:pre-commit', 'git add' ],
  lint-staged   './**/*.{ts,tsx}': [ 'yarn lint:pre-commit', 'git add' ] } +6ms
  lint-staged:cfg Normalizing config +0ms
  lint-staged:cfg Validating config +2ms
Running lint-staged with the following config:
{
  linters: {
    './**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx}': [
      'yarn format:pre-commit',
      'git add'
    ],
    './**/*.{ts,tsx}': [
      'yarn lint:pre-commit',
      'git add'
    ]
  },
  concurrent: true,
  chunkSize: 9007199254740991,
  globOptions: {
    matchBase: true,
    dot: true
  },
  ignore: [],
  subTaskConcurrency: 1,
  renderer: 'verbose',
  relative: false
}
  lint-staged:run Running all linter scripts +0ms
  lint-staged:run Resolved git directory to be `/path/to/project` +0ms
  lint-staged:run Loaded list of staged files in git:
  lint-staged:run [ 'package.json' ] +26ms
  lint-staged:gen-tasks Generating linter tasks +0ms
  lint-staged:cfg Normalizing config +30ms
  lint-staged:gen-tasks Generated task:
  lint-staged:gen-tasks { pattern: './**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx}',
  lint-staged:gen-tasks   commands: [ 'yarn format:pre-commit', 'git add' ],
  lint-staged:gen-tasks   fileList:
  lint-staged:gen-tasks    [ '/path/to/project/package.json' ] } +15ms
  lint-staged:gen-tasks Generated task:
  lint-staged:gen-tasks { pattern: './**/*.{ts,tsx}',
  lint-staged:gen-tasks   commands: [ 'yarn lint:pre-commit', 'git add' ],
  lint-staged:gen-tasks   fileList: [] } +2ms
Stashing changes... [started]
  lint-staged:git Stashing files... +0ms
  lint-staged:git Running git command [ 'write-tree' ] +0ms
  lint-staged:git Running git command [ 'add', '.' ] +16ms
  lint-staged:git Running git command [ 'write-tree' ] +20ms
  lint-staged:git Running git command [ 'read-tree', 'b0ec14163407bbc05f0579b3aae2f5caf31e97ba' ] +17ms
  lint-staged:git Running git command [ 'checkout-index', '-af' ] +20ms
  lint-staged:git Done stashing files! +39ms
Stashing changes... [completed]
Running linters... [started]
Running tasks for ./**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx} [started]
Running tasks for ./**/*.{ts,tsx} [started]
  lint-staged:make-cmd-tasks Creating listr tasks for commands [ 'yarn format:pre-commit', 'git add' ] +0ms
  lint-staged:find-bin Resolving binary for command `yarn format:pre-commit` +374ms
  lint-staged:find-bin Binary for `yarn format:pre-commit` resolved to `/Users/*****/.brew/bin/yarn` +6ms
  lint-staged:task ✔  OS: darwin; File path chunking unnecessary +0ms
  lint-staged:find-bin Resolving binary for command `git add` +0ms
  lint-staged:find-bin Binary for `git add` resolved to `/Users/*****/.brew/Cellar/git/2.20.1/libexec/git-core/git` +2ms
  lint-staged:task ✔  OS: darwin; File path chunking unnecessary +2ms
Running tasks for ./**/*.{ts,tsx} [skipped]
→ No staged files match ./**/*.{ts,tsx}
yarn format:pre-commit [started]
  lint-staged:task bin: /Users/*****/.brew/bin/yarn +8ms
  lint-staged:task args: [ 'format:pre-commit',
  lint-staged:task   '/path/to/project/package.json' ] +0ms
  lint-staged:task opts: { reject: false } +1ms
yarn format:pre-commit [completed]
git add [started]
  lint-staged:task bin: /Users/*****/.brew/Cellar/git/2.20.1/libexec/git-core/git +971ms
  lint-staged:task args: [ 'add',
  lint-staged:task   '/path/to/project/package.json' ] +0ms
  lint-staged:task opts: { reject: false } +0ms
git add [completed]
Running tasks for ./**/*.{js,jsx,ts,tsx,json,css,scss,html,md,mdx} [completed]
Running linters... [completed]
Updating stash... [started]
  lint-staged:git Running git command [ 'write-tree' ] +1s
Updating stash... [completed]
Restoring local changes... [started]
  lint-staged:git Restoring working copy +18ms
  lint-staged:git Running git command [ 'read-tree', '1c5f4eb6b89892241145971a0c2142a7ba066950' ] +0ms
  lint-staged:git Running git command [ 'checkout-index', '-af' ] +22ms
  lint-staged:git Restoring index with formatting changes +54ms
  lint-staged:git Running git command [ 'read-tree', 'b0ec14163407bbc05f0579b3aae2f5caf31e97ba' ] +1ms
  lint-staged:git Generating diff between trees b0ec14163407bbc05f0579b3aae2f5caf31e97ba and b0ec14163407bbc05f0579b3aae2f5caf31e97ba... +28ms
  lint-staged:git Running git command [ 'diff-tree',
  '--ignore-submodules',
  '--binary',
  '--no-color',
  '--no-ext-diff',
  '--unified=0',
  'b0ec14163407bbc05f0579b3aae2f5caf31e97ba',
  'b0ec14163407bbc05f0579b3aae2f5caf31e97ba' ] +0ms
  lint-staged:git Running git command [ 'apply',
  '-v',
  '--whitespace=nowarn',
  '--reject',
  '--recount',
  '--unidiff-zero' ] +22ms
  lint-staged:git Could not apply patch to the stashed files cleanly +25ms
  lint-staged:git Error: Error: Command failed: git apply -v --whitespace=nowarn --reject --recount --unidiff-zero
  lint-staged:git error: unrecognized input
  lint-staged:git
  lint-staged:git
  lint-staged:git     at execGit (/path/to/project/node_modules/lint-staged/src/gitWorkflow.js:28:11)
  lint-staged:git     at process.internalTickCallback (internal/process/next_tick.js:77:7) +1ms
  lint-staged:git Patch content: +0ms
  lint-staged:git
  lint-staged:git  +0ms
  lint-staged:git Found conflicts between formatters and local changes. Formatters changes will be ignored for conflicted hunks. +0ms
  lint-staged:git Deleted files and folders:
  lint-staged:git   +6ms
Restoring local changes... [completed]
  lint-staged linters were executed successfully! +1s
[develop ac01d35] trigger lint-staged debug log
 1 file changed, 1 insertion(+), 1 deletion(-)

Environment

• OS: macOS Mojave 10.14.2
• Node.js: v11.4.0
• yarn: v1.12.3
• lint-staged: v8.1.0
• husky: v1.1.3

[tremby]

I was just testing lint-staged as an alternative to the shell script I've been using for the same purpose.

I noticed the same thing: a tracked file which has been deleted from the work tree but whose deletion has not been staged get undeleted as part of lint-staged's operation.

Maintainers, see my script at https://stackoverflow.com/a/26685296/496046 for how I worked around this issue. It's pretty basic -- I just took a list of these files and then deleted them again when finished.

[tremby]

I tested a little further.

In all four of these cases: (git status --porcelain-style status key given for each)

• Staged addition of a file, but then unstaged deletion (AD)
• No staged change, but unstaged deletion ( D)
• Staged rename of a file, but then unstaged deletion (RD)
• Staged modification of a file, but then unstaged deletion (MD)

The file should be deleted when lint-staged is done, but it exists in the work tree again.

[okonet]

🎉 This issue has been resolved in version 9.5.0-beta.1 🎉

The release is available on:

• npm package (@beta dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[tremby]

This is fixed for the MD and  D cases, but not for the AD or RD cases.

Here's a tarball of a repo in a state where files are in all four of the above statuses, plus one staged file which lint-staged takes into account.

Test repo with various things staged/deleted

If you extract this and cd into it, run git status --porcelain to see the following:

AD fileAD
MD fileMD
RD fileRD -> fileRD_renamed
 D file_D
A  test.js

If you run npx lint-staged, it will do its stashing etc and then (successfully) run prettier on the javascript file, but the state it leaves things in is not the same as the state it started in. Running git status --porcelain again gives

A  fileAD
MD fileMD
R  fileRD -> fileRD_renamed
 D file_D
A  test.js

The files fileRD_renamed and fileAD should not be present in the work tree at this point, but are. You then need to run rm fileRD_renamed fileAD to return to the desired state.

Here's a ttyrec session of me setting this repo up from scratch and testing it. I make a couple of mistakes in my rm commands at the end, but hopefully it still makes sense.

ttyrec recording (play with ttyplay lint-staged-test.txt)

[okonet]

🎉 This issue has been resolved in version 10.0.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[tremby]

I updated to 10.0.1 and this is still not fixed for the AD and RD cases. I see exactly the same results as posted in my previous message in this thread.

Here's a new git repo tarball with lint-staged updated:
lint-staged-test-2.tar.gz

$ git status --porcelain
AD fileAD
MD fileMD
RD fileRD -> fileRD_renamed
 D file_D
A  test.js
$ npx lint-staged
  ✔ Preparing...
  ✔ Running tasks...
  ✔ Applying modifications...
  ✔ Cleaning up...
$ git status --porcelain
A  fileAD
MD fileMD
R  fileRD -> fileRD_renamed
 D file_D
A  test.js
$

I then need to run rm fileAD fileRD_renamed to get it back to the state it was in before lint-staged ran.

[iiroj]

Thanks @tremby. Let me open this ticket again and see.

EDIT1: This seems to be an issue in the calculation of the unstaged diff:

diff --git b/fileMD a/fileMD
deleted file mode 100644
index 031943e..0000000
--- b/fileMD
+++ /dev/null
@@ -1,2 +0,0 @@
-fileMD
-new content
diff --git b/file_D a/file_D
deleted file mode 100644
index c97750d..0000000
--- b/file_D
+++ /dev/null
@@ -1 +0,0 @@
-file_D

EDIT2: This is the backup stash, that should include the original state. It seems to miss the AD and RD cases:

new file mode 100644
index 0000000..fa49b07
--- /dev/null
+++ b/fileAD
@@ -0,0 +1 @@
+new file
diff --git a/fileMD b/fileMD
deleted file mode 100644
index 3c085e3..0000000
--- a/fileMD
+++ /dev/null
@@ -1 +0,0 @@
-fileMD
diff --git a/fileRD b/fileRD_renamed
similarity index 100%
rename from fileRD
rename to fileRD_renamed
diff --git a/file_D b/file_D
deleted file mode 100644
index c97750d..0000000
--- a/file_D
+++ /dev/null
@@ -1 +0,0 @@
-file_D
diff --git a/test.js b/test.js
new file mode 100644
index 0000000..4636d04
--- /dev/null
+++ b/test.js
@@ -0,0 +1,3 @@

EDIT3: This issue seems to come from git stash itself. In your repo, you will end up with the same situation if you run these:

git stash save --include-untracked
git stash pop

[iiroj]

Maybe this would help with the deleted files reappearing:

❯ git ls-files --deleted         
fileAD
fileMD
fileRD_renamed
file_D

❯ npx lint-staged
  ✔ Preparing...
  ✔ Running tasks...
  ✔ Applying modifications...
  ✔ Cleaning up...

❯ git ls-files --deleted 
fileMD
file_D

Save the first list and delete them after running?

[iiroj]

@tremby Do you think you could check out the code at #778 and see if it's finally fixed. At least the git status in your tarball is stabilized now.

[tremby]

@iiroj I dropped some comments over on that PR. Works for me! Thanks!

[okonet]

🎉 This issue has been resolved in version 10.0.6 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀