Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lint-staged shows a different set of files to be committed than actually gets committed, and is not stashing changes before hand #1346

Open
trusktr opened this issue Oct 31, 2023 · 4 comments

Comments

@trusktr
Copy link

trusktr commented Oct 31, 2023

And it also does not appear to stash things before doing anything.

 

Description

I have this in husky pre-commit:

#!/usr/bin/env sh
. "$(dirname -- "$0")/_/husky.sh"

npx lint-staged

Here's lintstagedrc:

const buildSteps = ['git reset dist', 'git checkout dist', 'npm run build', 'git status', 'git add dist']

export default {
	'./src/**/*': () => buildSteps
}

What I'm trying to do is add build outputs if src/ files changed.

Steps to reproduce

The repo is here: https://github.com/lume/lume, in the develop branch. See .lintstagedrc.js

git clone git@github.com:lume/lume.git
cd lume
git checkout d90b8449d92391faa61507d71aa31a5449b6cbd5
npm run fresh
# modify a few src/ files (stick console.logs in them)
# now also modify any file inside of apps/docs/, just add a comment anywhere, doesn't matter what you add
# note that apps/docs/ is a git submodule, and submodule changes cannot be stashed in the parent
npm run build
git status #shows changes to both src/ and dist/
git add src/one/of/the/files.ts # add only one of the files
git status # shows one file in src/ is staged
git commit
# in the editor that opens, it shows only the src/ file staged
# write a message, save and exit the editor

Now, inspect the diff for the commit, and it will also contained modified dist/ files, including files corresponding to all src/ files that were modified, not just the one that was committed.

If I have this in my working dir:

❯ git status                                                                                                                                                                    ✹ ✚
On branch develop
Your branch is up to date with 'origin/develop'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        modified:   src/cameras/CameraRig.ts
        modified:   src/examples/FlickeringOrb.ts
        modified:   src/examples/FlickeringOrbs.ts

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
  (commit or discard the untracked or modified content in submodules)
        modified:   .husky/pre-commit
        modified:   .lintstagedrc.js
        modified:   README.md
        modified:   apps/docs (modified content)
        modified:   dist/behaviors/PropReceiver.d.ts.map
        modified:   dist/behaviors/PropReceiver.js.map
        modified:   dist/cameras/CameraRig.d.ts
        modified:   dist/cameras/CameraRig.d.ts.map
        modified:   dist/cameras/CameraRig.js
        modified:   dist/cameras/CameraRig.js.map
        modified:   dist/examples/FlickeringOrb.d.ts
        modified:   dist/examples/FlickeringOrb.d.ts.map
        modified:   dist/examples/FlickeringOrb.js
        modified:   dist/examples/FlickeringOrb.js.map
        modified:   dist/examples/FlickeringOrbs.d.ts
        modified:   dist/examples/FlickeringOrbs.d.ts.map
        modified:   dist/examples/FlickeringOrbs.js
        modified:   dist/examples/FlickeringOrbs.js.map
        modified:   dist/lights/PointLight.d.ts.map
        modified:   dist/lights/PointLight.js.map
        modified:   src/behaviors/PropReceiver.ts
        modified:   src/behaviors/mesh-behaviors/ClipPlanesBehavior.ts
        modified:   src/lights/PointLight.ts
        modified:   src/lights/SpotLight.ts

then when npx lint-staged runs, it tells me the same thing when I edit the commit message (as if it will commit only src but not dist.

However, when I go inspect the commit itself, I see the equivalent of having committed this:

❯ git status                                                                                                                                                                    ✹ ✚
On branch develop
Your branch is up to date with 'origin/develop'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        modified:   dist/behaviors/PropReceiver.d.ts.map
        modified:   dist/behaviors/PropReceiver.js.map
        modified:   dist/cameras/CameraRig.d.ts
        modified:   dist/cameras/CameraRig.d.ts.map
        modified:   dist/cameras/CameraRig.js
        modified:   dist/cameras/CameraRig.js.map
        modified:   dist/examples/FlickeringOrb.d.ts
        modified:   dist/examples/FlickeringOrb.d.ts.map
        modified:   dist/examples/FlickeringOrb.js
        modified:   dist/examples/FlickeringOrb.js.map
        modified:   dist/examples/FlickeringOrbs.d.ts
        modified:   dist/examples/FlickeringOrbs.d.ts.map
        modified:   dist/examples/FlickeringOrbs.js
        modified:   dist/examples/FlickeringOrbs.js.map
        modified:   dist/lights/PointLight.d.ts.map
        modified:   dist/lights/PointLight.js.map
        modified:   src/cameras/CameraRig.ts
        modified:   src/examples/FlickeringOrb.ts
        modified:   src/examples/FlickeringOrbs.ts

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
  (commit or discard the untracked or modified content in submodules)
        modified:   .husky/pre-commit
        modified:   .lintstagedrc.js
        modified:   README.md
        modified:   apps/docs (modified content)
        modified:   src/behaviors/PropReceiver.ts
        modified:   src/behaviors/mesh-behaviors/ClipPlanesBehavior.ts
        modified:   src/lights/PointLight.ts
        modified:   src/lights/SpotLight.ts

This indicates that dist/ is being added, just as I've specified in my buildSteps.

However, we see that it includes dist/behaviors/PropReceiver.js.map.

But if lint-staged is supposed to be stashing things first, then that output file should not exist because src/behaviors/PropReceiver.ts would have been stashed, therefore npm run build will not modify that file in dist/.

I can verify that if I stash changes manually myself, then run npm run build, no changes happen to PropReceiver.* inside of dist/, as expected, so this indicates that lint-staged is running my buildSteps with the unstaged files in place.

Debug Logs

expand to view
❯ npx lint-staged --debug                                                                                                                                                       ✹ ✚
  lint-staged:bin Options parsed from command-line: {
  allowEmpty: false,
  concurrent: true,
  configPath: undefined,
  cwd: undefined,
  debug: true,
  diff: undefined,
  diffFilter: undefined,
  maxArgLength: undefined,
  quiet: false,
  relative: false,
  shell: false,
  stash: true,
  verbose: false
} +0ms
  lint-staged:validateOptions Validating options... +0ms
  lint-staged:validateOptions Validated options! +0ms
  lint-staged Unset GIT_LITERAL_PATHSPECS (was `undefined`) +0ms
  lint-staged:runAll Running all linter scripts... +0ms
  lint-staged:runAll Using working directory `/Users/trusktr/src/lume+lume+develop` +0ms
  lint-staged:resolveGitRepo Resolving git repo from `/Users/trusktr/src/lume+lume+develop` +0ms
  lint-staged:resolveGitRepo Unset GIT_DIR (was `undefined`) +0ms
  lint-staged:resolveGitRepo Unset GIT_WORK_TREE (was `undefined`) +0ms
  lint-staged:execGit Running git command [ 'rev-parse', '--show-prefix' ] +0ms
  lint-staged:resolveGitRepo Resolved git directory to be `/Users/trusktr/src/lume+lume+develop` +43ms
  lint-staged:resolveGitRepo Resolved git config directory to be `/Users/trusktr/src/lume+lume+develop/.git` +0ms
  lint-staged:execGit Running git command [ 'log', '-1' ] +43ms
  lint-staged:execGit Running git command [ 'diff', '--name-only', '-z', '--diff-filter=ACMR', '--staged' ] +53ms
  lint-staged:runAll Loaded list of staged files in git:
  lint-staged:runAll [
  lint-staged:runAll   '/Users/trusktr/src/lume+lume+develop/.lintstagedrc.js',
  lint-staged:runAll   '/Users/trusktr/src/lume+lume+develop/src/cameras/CameraRig.ts',
  lint-staged:runAll   '/Users/trusktr/src/lume+lume+develop/src/examples/FlickeringOrb.ts',
  lint-staged:runAll   '/Users/trusktr/src/lume+lume+develop/src/examples/FlickeringOrbs.ts'
  lint-staged:runAll ] +131ms
  lint-staged:searchConfigs Searching for configuration files... +0ms
  lint-staged:execGit Running git command [ 'ls-files', '-z', '--full-name' ] +36ms
  lint-staged:execGit Running git command [ 'ls-files', '-z', '--full-name', '--others', '--exclude-standard' ] +41ms
  lint-staged:searchConfigs Found possible config files: [
  '/Users/trusktr/src/lume+lume+develop/examples/react-typescript/package.json',
  '/Users/trusktr/src/lume+lume+develop/examples/react/package.json',
  '/Users/trusktr/src/lume+lume+develop/.lintstagedrc.js',
  '/Users/trusktr/src/lume+lume+develop/package.json'
] +71ms
  lint-staged:loadConfig Loading configuration from `/Users/trusktr/src/lume+lume+develop/examples/react-typescript/package.json`... +0ms
  lint-staged:loadConfig Loading configuration from `/Users/trusktr/src/lume+lume+develop/examples/react/package.json`... +0ms
  lint-staged:loadConfig Loading configuration from `/Users/trusktr/src/lume+lume+develop/.lintstagedrc.js`... +0ms
  lint-staged:loadConfig Loading configuration from `/Users/trusktr/src/lume+lume+develop/package.json`... +1ms
  lint-staged:loadConfig Successfully loaded config from `/Users/trusktr/src/lume+lume+develop/package.json`:
  lint-staged:loadConfig null +0ms
  lint-staged:loadConfig Successfully loaded config from `/Users/trusktr/src/lume+lume+develop/examples/react/package.json`:
  lint-staged:loadConfig null +1ms
  lint-staged:loadConfig Successfully loaded config from `/Users/trusktr/src/lume+lume+develop/examples/react-typescript/package.json`:
  lint-staged:loadConfig null +0ms
  lint-staged:loadConfig Successfully loaded config from `/Users/trusktr/src/lume+lume+develop/.lintstagedrc.js`:
  lint-staged:loadConfig {
  lint-staged:loadConfig   './src/**/*': [Function: ./src/**/*],
  lint-staged:loadConfig   './packages/**/*': [Function: ./packages/**/*]
  lint-staged:loadConfig } +0ms
  lint-staged:validateConfig Validating config from `/Users/trusktr/src/lume+lume+develop/.lintstagedrc.js`... +0ms
  lint-staged:validateConfig Validated config from `/Users/trusktr/src/lume+lume+develop/.lintstagedrc.js`: +0ms
  lint-staged:validateConfig {
  lint-staged:validateConfig   './src/**/*': [Function: ./src/**/*],
  lint-staged:validateConfig   './packages/**/*': [Function: ./packages/**/*]
  lint-staged:validateConfig } +0ms
  lint-staged:searchConfigs Found 1 config files +2ms
  lint-staged:groupFilesByConfig Grouping 4 files by 1 configurations +0ms
  lint-staged:chunkFiles Resolved an argument string length of 250 characters from 4 files +0ms
  lint-staged:chunkFiles Creating 1 chunks for maxArgLength of 131072 +0ms
  lint-staged:generateTasks Generating linter tasks +0ms
  lint-staged:generateTasks Generated task: 
  lint-staged:generateTasks {
  lint-staged:generateTasks   pattern: './src/**/*',
  lint-staged:generateTasks   commands: [Function: ./src/**/*],
  lint-staged:generateTasks   fileList: [
  lint-staged:generateTasks     '/Users/trusktr/src/lume+lume+develop/src/cameras/CameraRig.ts',
  lint-staged:generateTasks     '/Users/trusktr/src/lume+lume+develop/src/examples/FlickeringOrb.ts',
  lint-staged:generateTasks     '/Users/trusktr/src/lume+lume+develop/src/examples/FlickeringOrbs.ts'
  lint-staged:generateTasks   ]
  lint-staged:generateTasks } +1ms
  lint-staged:generateTasks Generated task: 
  lint-staged:generateTasks {
  lint-staged:generateTasks   pattern: './packages/**/*',
  lint-staged:generateTasks   commands: [Function: ./packages/**/*],
  lint-staged:generateTasks   fileList: []
  lint-staged:generateTasks } +1ms
  lint-staged:makeCmdTasks Creating listr tasks for commands [Function: ./src/**/*] +0ms
  lint-staged:makeCmdTasks Creating listr tasks for commands [Function: ./packages/**/*] +0ms
  lint-staged:resolveTaskFn cmd: git +0ms
  lint-staged:resolveTaskFn args: [ 'reset', 'dist' ] +0ms
  lint-staged:resolveTaskFn execaOptions: {
  cwd: '/Users/trusktr/src/lume+lume+develop',
  preferLocal: true,
  reject: false,
  shell: false
} +0ms
  lint-staged:resolveTaskFn cmd: git +0ms
  lint-staged:resolveTaskFn args: [ 'checkout', 'dist' ] +0ms
  lint-staged:resolveTaskFn execaOptions: {
  cwd: '/Users/trusktr/src/lume+lume+develop',
  preferLocal: true,
  reject: false,
  shell: false
} +0ms
  lint-staged:resolveTaskFn cmd: npm +0ms
  lint-staged:resolveTaskFn args: [ 'run', 'build' ] +0ms
  lint-staged:resolveTaskFn execaOptions: {
  cwd: '/Users/trusktr/src/lume+lume+develop',
  preferLocal: true,
  reject: false,
  shell: false
} +0ms
  lint-staged:resolveTaskFn cmd: git +0ms
  lint-staged:resolveTaskFn args: [ 'status' ] +0ms
  lint-staged:resolveTaskFn execaOptions: {
  cwd: '/Users/trusktr/src/lume+lume+develop',
  preferLocal: true,
  reject: false,
  shell: false
} +0ms
  lint-staged:resolveTaskFn cmd: git +0ms
  lint-staged:resolveTaskFn args: [ 'add', 'dist' ] +0ms
  lint-staged:resolveTaskFn execaOptions: {
  cwd: '/Users/trusktr/src/lume+lume+develop',
  preferLocal: true,
  reject: false,
  shell: false
} +0ms
  lint-staged:resolveTaskFn cmd: git +0ms
  lint-staged:resolveTaskFn args: [ 'reset', 'dist' ] +0ms
  lint-staged:resolveTaskFn execaOptions: {
  cwd: '/Users/trusktr/src/lume+lume+develop',
  preferLocal: true,
  reject: false,
  shell: false
} +0ms
  lint-staged:resolveTaskFn cmd: git +1ms
  lint-staged:resolveTaskFn args: [ 'checkout', 'dist' ] +0ms
  lint-staged:resolveTaskFn execaOptions: {
  cwd: '/Users/trusktr/src/lume+lume+develop',
  preferLocal: true,
  reject: false,
  shell: false
} +0ms
  lint-staged:resolveTaskFn cmd: npm +0ms
  lint-staged:resolveTaskFn args: [ 'run', 'build' ] +0ms
  lint-staged:resolveTaskFn execaOptions: {
  cwd: '/Users/trusktr/src/lume+lume+develop',
  preferLocal: true,
  reject: false,
  shell: false
} +0ms
  lint-staged:resolveTaskFn cmd: git +0ms
  lint-staged:resolveTaskFn args: [ 'status' ] +0ms
  lint-staged:resolveTaskFn execaOptions: {
  cwd: '/Users/trusktr/src/lume+lume+develop',
  preferLocal: true,
  reject: false,
  shell: false
} +0ms
  lint-staged:resolveTaskFn cmd: git +0ms
  lint-staged:resolveTaskFn args: [ 'add', 'dist' ] +0ms
  lint-staged:resolveTaskFn execaOptions: {
  cwd: '/Users/trusktr/src/lume+lume+develop',
  preferLocal: true,
  reject: false,
  shell: false
} +0ms
  lint-staged:chunkFiles Resolved an argument string length of 196 characters from 3 files +3ms
  lint-staged:chunkFiles Creating 1 chunks for maxArgLength of 131072 +0ms
[STARTED] Preparing lint-staged...
  lint-staged:GitWorkflow Backing up original state... +0ms
  lint-staged:GitWorkflow Getting partially staged files... +0ms
  lint-staged:execGit Running git command [ 'status', '-z' ] +38ms
  lint-staged:GitWorkflow Found partially staged files: [] +173ms
  lint-staged:GitWorkflow Backing up merge state... +0ms
  lint-staged:file Reading file `/Users/trusktr/src/lume+lume+develop/.git/MERGE_HEAD` +0ms
  lint-staged:file Reading file `/Users/trusktr/src/lume+lume+develop/.git/MERGE_MODE` +1ms
  lint-staged:file Reading file `/Users/trusktr/src/lume+lume+develop/.git/MERGE_MSG` +0ms
  lint-staged:file File `/Users/trusktr/src/lume+lume+develop/.git/MERGE_HEAD` doesn't exist, ignoring... +0ms
  lint-staged:file File `/Users/trusktr/src/lume+lume+develop/.git/MERGE_MODE` doesn't exist, ignoring... +0ms
  lint-staged:file File `/Users/trusktr/src/lume+lume+develop/.git/MERGE_MSG` doesn't exist, ignoring... +0ms
  lint-staged:GitWorkflow Done backing up merge state! +1ms
  lint-staged:GitWorkflow Getting deleted files... +0ms
  lint-staged:execGit Running git command [ 'ls-files', '--deleted' ] +173ms
  lint-staged:GitWorkflow Found deleted files: [] +46ms
  lint-staged:execGit Running git command [ 'stash', 'create' ] +46ms
  lint-staged:execGit Running git command [
  'stash',
  'store',
  '--quiet',
  '--message',
  'lint-staged automatic backup',
  '30f471090880fac647657ababa2b5b29e622f555'
] +208ms
  lint-staged:GitWorkflow Done backing up original state! +231ms
[COMPLETED] Preparing lint-staged...
[STARTED] Running tasks for staged files...
[STARTED] .lintstagedrc.js — 4 files
[STARTED] ./src/**/* — 3 files
[STARTED] ./packages/**/* — 0 files
[SKIPPED] ./packages/**/* — no files
[STARTED] git reset dist
[COMPLETED] git reset dist
[STARTED] git checkout dist
[COMPLETED] git checkout dist
[STARTED] npm run build
[COMPLETED] npm run build
[STARTED] git status
[COMPLETED] git status
[STARTED] git add dist
[COMPLETED] git add dist
[COMPLETED] ./src/**/* — 3 files
[COMPLETED] .lintstagedrc.js — 4 files
[COMPLETED] Running tasks for staged files...
[STARTED] Applying modifications from tasks...
  lint-staged:GitWorkflow Adding task modifications to index... +4s
  lint-staged:execGit Running git command [
  'add',
  '--',
  '/Users/trusktr/src/lume+lume+develop/src/cameras/CameraRig.ts',
  '/Users/trusktr/src/lume+lume+develop/src/examples/FlickeringOrb.ts',
  '/Users/trusktr/src/lume+lume+develop/src/examples/FlickeringOrbs.ts'
] +4s
  lint-staged:GitWorkflow Done adding task modifications to index! +41ms
  lint-staged:execGit Running git command [ 'diff', '--name-only', '-z', '--diff-filter=ACMR', '--staged' ] +41ms
[COMPLETED] Applying modifications from tasks...
[STARTED] Cleaning up temporary files...
  lint-staged:GitWorkflow Dropping backup stash... +35ms
  lint-staged:execGit Running git command [ 'stash', 'list' ] +35ms
  lint-staged:execGit Running git command [ 'stash', 'drop', '--quiet', '0' ] +37ms
  lint-staged:GitWorkflow Done dropping backup stash! +79ms
[COMPLETED] Cleaning up temporary files...
  lint-staged Tasks were executed successfully! +5s

Environment

  • OS: macoS
  • Node.js: 20
  • lint-staged: 14.0.1
@trusktr trusktr changed the title lint-staged shows a different set of files to be committed than actually gets committed. lint-staged shows a different set of files to be committed than actually gets committed, and is not stashing changes before hand Oct 31, 2023
@trusktr
Copy link
Author

trusktr commented Oct 31, 2023

I see that the debug output runs this:

  lint-staged:execGit Running git command [ 'stash', 'create' ] +46ms
  lint-staged:execGit Running git command [
  'stash',
  'store',
  '--quiet',
  '--message',
  'lint-staged automatic backup',
  '30f471090880fac647657ababa2b5b29e622f555'
] +208ms
  lint-staged:GitWorkflow Done backing up original state! +231ms

It claims that it stashed things. Is that step silently failing to work, with a false-positive message?

@trusktr
Copy link
Author

trusktr commented Oct 31, 2023

I also have modifications in git submodules. Could that have to do with it? I know that git submodule changes cannot be stashed in the parent. I've updated the repro to include a step to modify a git submodule.

@trusktr
Copy link
Author

trusktr commented Oct 31, 2023

I can't figure out what the issue is with lint-staged, but for now I was able to work around it in a less ideal way like so:

First, change .lintstagedrc.js to this:

const buildSteps = ['npm run build', 'git add dist']

export default {
	'./src/**/*': () => buildSteps,
	'./packages/**/*': () => buildSteps,
}

Then change .husky/pre-commit to this:

#!/usr/bin/env sh
. "$(dirname -- "$0")/_/husky.sh"

# If a previous commit attempt's stash still exists, notify, and exit.
(git stash list | grep COMMIT_HOOK_STASH) && echo "A previous git stash named "COMMIT_HOOK_STASH" exists. Please handle and remove the stash then try again." && exit 1

git reset dist
git checkout dist
git stash save -k COMMIT_HOOK_STASH
npx lint-staged

Then install .husky/post-commit with this in it:

#!/usr/bin/env sh
. "$(dirname -- "$0")/_/husky.sh"

# unstash only if there was a stash
(git stash list | grep COMMIT_HOOK_STASH) && git stash pop

This basically does what I was trying to do with lint-staged, but it is not as ideal as it would be with lint-staged because:

  • it applies the revert of dist/ no matter which files changed, instead of based on a glob like with lint-staged, but it's not the end of the world, and if lint-staged were doing what I was expecting, then it would revert all dist/ change anyway, so a re-build is needed anyway after a commit of src/ files.
  • manual handling of stashes is not as robust (that's why we want to rely on lint-staged), we need to remember to remove the COMMIT_HOOK_STASH if there was a failure, otherwise the next commit attempt will fail.

lint-staged has been battle hardened to handle so many edge cases, while my workaround is a naive setup that I'm trying to avoid by using lint-staged in the first place. I'm happy for my workaround around though, now I can move forward for now, while still using lint-staged to apply other things like prettier/linting to staged files, etc. But it would be great to be able to do what I'm trying to do with the dist/ folder using lint-staged.

@iiroj
Copy link
Member

iiroj commented Nov 1, 2023

Lint-staged uses git stash create and git stash store to create the backup stash. It does not use the more common git stash push which also checks out the changes. So basically the stash is created but the staged/modified files do not change on disk. Only "partially staged" files are reset when running the tasks, and lint-staged tries to avoid doing anything to files that are not staged, or did not exist before starting.

With this in mind, does it make more sense?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants