Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NX Passed 11161 file paths to Git to hash, but received 11148 hashes. #9946

Closed
jonhamm opened this issue Apr 22, 2022 · 7 comments · Fixed by #10134
Closed

NX Passed 11161 file paths to Git to hash, but received 11148 hashes. #9946

jonhamm opened this issue Apr 22, 2022 · 7 comments · Fixed by #10134
Labels

Comments

@jonhamm
Copy link
Contributor

jonhamm commented Apr 22, 2022

Current Behavior

This is effectively a re-open of : #8779

After upgrading

  • node from 14.16.1 to 16.14.2
  • npm from 6.14.12 to 8.5.0
  • nx from 13.8.0 to 13.10.3

I have frequent builds on my Jenkins builds failing with errors like
> NX Passed 11161 file paths to Git to hash, but received 11148 hashes.

Expected Behavior

  • The hashing should log the error returned from git hash-object e.g. fatal: Cannot open 'xxxx': No such file or directory
  • If the hashing fails builds should not fail - i.e. the build should proceed nx caching disabled

Steps to Reproduce

Have not seen the pattern that causes these errors to occur
The proposed improvement in "Expected Behaviour" will not solve the problem but will

  • provide more info that could make it possibly solve the error - e.g. from > NX Passed 7460 file paths to Git to hash, but received 1709 hashes. #8779 one solved the problem by adding a directory with temporary files to .gitignore and this might also be a solution to my problem, but which directory ??
  • mitigate the effect of the problem by allowing the build to complete albeit without caching

Failure Logs

Environment

NX Report complete - copy this into the issue template

   Node : 16.14.2
   OS   : darwin x64
   npm  : 8.5.0
   
   nx : 13.10.3
   @nrwl/angular : 13.9.5
   @nrwl/cypress : 13.9.5
   @nrwl/detox : Not Found
   @nrwl/devkit : 13.9.5
   @nrwl/eslint-plugin-nx : 13.9.5
   @nrwl/express : Not Found
   @nrwl/jest : 13.9.5
   @nrwl/js : Not Found
   @nrwl/linter : 13.9.5
   @nrwl/nest : Not Found
   @nrwl/next : Not Found
   @nrwl/node : Not Found
   @nrwl/nx-cloud : 13.2.1
   @nrwl/nx-plugin : Not Found
   @nrwl/react : Not Found
   @nrwl/react-native : Not Found
   @nrwl/schematics : Not Found
   @nrwl/storybook : 13.9.5
   @nrwl/web : Not Found
   @nrwl/workspace : 13.9.5
   typescript : 4.5.5
   rxjs : 6.6.7
   ---------------------------------------
   Community plugins:
   	 @ngrx/component-store: 13.0.2
   	 @ngrx/store: 13.0.2
   	 @ngx-matomo/tracker: 2.4.1
   	 apollo-angular: 2.6.0
   	 @compodoc/compodoc: 1.1.19
   	 @storybook/angular: 6.4.22
   	 @testing-library/angular: 10.11.1
   	 ng-mocks: 13.4.2
   	 nx-stylelint: 12.3.1

@AgentEnder AgentEnder added the scope: core core nx functionality label Apr 22, 2022
@jonhamm
Copy link
Contributor Author

jonhamm commented Apr 26, 2022

It appears to be related to having run-commands execute nx run-many ... --parallel

   "%lint": {
      "executor": "@nrwl/workspace:run-commands",
      "options": {
        "parallel": false,
        "commands": [
          "npx env-cmd -e nx-not-runner-invoked npx nx test tools",
          "npx env-cmd -e nx-not-runner-invoked npx nx run-many --all --target=lint --parallel --exclude=common"
        ]
      }
    },
 

@jonhamm
Copy link
Contributor Author

jonhamm commented Apr 28, 2022

I now realised this is a node 16.14.2 induced problem

In this setting I do get the error immediately


 >  NX   Report complete - copy this into the issue template

   Node : 16.14.2
   OS   : darwin x64
   npm  : 8.5.0
   
   nx : 13.8.0
   @nrwl/angular : 13.8.0
   @nrwl/cli : 13.8.0
   @nrwl/cypress : 13.8.0
   @nrwl/detox : undefined
   @nrwl/devkit : 13.8.0
   @nrwl/eslint-plugin-nx : 13.8.0
   @nrwl/express : undefined
   @nrwl/jest : 13.8.0
   @nrwl/js : undefined
   @nrwl/linter : 13.8.0
   @nrwl/nest : undefined
   @nrwl/next : undefined
   @nrwl/node : undefined
   @nrwl/nx-cloud : 13.1.4
   @nrwl/react : undefined
   @nrwl/react-native : undefined
   @nrwl/schematics : undefined
   @nrwl/storybook : 13.8.0
   @nrwl/tao : 13.8.0
   @nrwl/web : undefined
   @nrwl/workspace : 13.8.0
   typescript : 4.5.5
   rxjs : 6.6.7
   ---------------------------------------
   Community plugins:
   	 @angular/animations: 13.2.2
   	 @angular/cdk: 13.2.1
   	 @angular/common: 13.2.2
   	 @angular/compiler: 13.2.2
   	 @angular/core: 13.2.2
   	 @angular/forms: 13.2.2
   	 @angular/platform-browser: 13.2.2
   	 @angular/platform-browser-dynamic: 13.2.2
   	 @angular/router: 13.2.2
   	 @angular/upgrade: 13.2.2
   	 @ngrx/component-store: 13.0.2
   	 @ngrx/store: 13.0.2
   	 @ngx-matomo/tracker: 2.4.1
   	 apollo-angular: 2.6.0
   	 @angular-builders/custom-webpack: 13.1.0
   	 @angular-devkit/build-angular: 13.2.2
   	 @angular/cli: 13.2.2
   	 @angular/compiler-cli: 13.2.2
   	 @angular/elements: 13.2.2
   	 @angular/language-service: 13.2.2
   	 @compodoc/compodoc: 1.1.19
   	 @storybook/angular: 6.4.22
   	 @testing-library/angular: 10.11.1
   	 ng-mocks: 13.4.2
   	 nx-stylelint: 12.3.1

But in this setting which is similar except for the node version there is no problem


 >  NX   Report complete - copy this into the issue template

   Node : 14.17.6
   OS   : darwin x64
   npm  : 6.14.15
   
   nx : undefined
   @nrwl/angular : 13.8.0
   @nrwl/cli : 13.8.0
   @nrwl/cypress : 13.8.0
   @nrwl/detox : undefined
   @nrwl/devkit : 13.8.0
   @nrwl/eslint-plugin-nx : 13.8.0
   @nrwl/express : undefined
   @nrwl/jest : 13.8.0
   @nrwl/js : undefined
   @nrwl/linter : 13.8.0
   @nrwl/nest : undefined
   @nrwl/next : undefined
   @nrwl/node : undefined
   @nrwl/nx-cloud : 13.1.4
   @nrwl/react : undefined
   @nrwl/react-native : undefined
   @nrwl/schematics : undefined
   @nrwl/storybook : 13.8.0
   @nrwl/tao : 13.8.0
   @nrwl/web : undefined
   @nrwl/workspace : 13.8.0
   typescript : 4.5.5
   rxjs : 6.6.7
   ---------------------------------------
   Community plugins:
   	 @angular/animations: 13.2.2
   	 @angular/cdk: 13.2.1
   	 @angular/common: 13.2.2
   	 @angular/compiler: 13.2.2
   	 @angular/core: 13.2.2
   	 @angular/forms: 13.2.2
   	 @angular/platform-browser: 13.2.2
   	 @angular/platform-browser-dynamic: 13.2.2
   	 @angular/router: 13.2.2
   	 @angular/upgrade: 13.2.2
   	 @ngrx/component-store: 13.0.2
   	 @ngrx/store: 13.0.2
   	 @ngx-matomo/tracker: 2.4.1
   	 apollo-angular: 2.6.0
   	 @angular-builders/custom-webpack: 13.1.0
   	 @angular-devkit/build-angular: 13.2.2
   	 @angular/cli: 13.2.2
   	 @angular/compiler-cli: 13.2.2
   	 @angular/elements: 13.2.2
   	 @angular/language-service: 13.2.2
   	 @compodoc/compodoc: 1.1.19
   	 @storybook/angular: 6.4.22
   	 @testing-library/angular: 10.11.1
   	 ng-mocks: 13.4.2
   	 nx-stylelint: 12.3.1


@jonhamm
Copy link
Contributor Author

jonhamm commented May 3, 2022

Turned out the problem was that .npm was not in .gitignore
It would however be very useful to get the stderr output from the spawned child process git hash-object ..... as this would have revealed this problem immediately so I will leave the issue open...

jonhamm added a commit to jonhamm/nx that referenced this issue May 4, 2022
…og one of the missing files

When executing parallel targets using e.g. run-commands missing entries in .gitignore
may lead to
temp files being passed to "git hash-object"
which when git executes have been deleted
resulting in
mismatch between number of files requested and hashes returned.
git reports the error and this PR
will make sure that report is passed
in the error thrown.
This will make problem resolution a lot
faster.

ISSUES CLOSED: nrwl#9946
jonhamm added a commit to jonhamm/nx that referenced this issue May 5, 2022
…og one of the missing files

When executing parallel targets using e.g. run-commands missing entries in .gitignore
may lead to
temp files being passed to "git hash-object"
which when git executes have been deleted
resulting in
mismatch between number of files requested and hashes returned.
git reports the error and this PR
will make sure that report is passed
in the error thrown.
This will make problem resolution a lot
faster.

ISSUES CLOSED: nrwl#9946
FrozenPandaz pushed a commit that referenced this issue May 6, 2022
…og one of the missing files (#10134)

When executing parallel targets using e.g. run-commands missing entries in .gitignore
may lead to
temp files being passed to "git hash-object"
which when git executes have been deleted
resulting in
mismatch between number of files requested and hashes returned.
git reports the error and this PR
will make sure that report is passed
in the error thrown.
This will make problem resolution a lot
faster.

ISSUES CLOSED: #9946
@julianpoemp
Copy link

julianpoemp commented May 19, 2022

I still get this error using Nx v14.1.7, Node v16.15 . Do you know how to fix this? The only workaround that I found is to restart my computer...

My error message:

nx serve api                
nx run [project][:target][:configuration] [_..]

Run a target for a project
(e.g., nx run myapp:serve:production).

You can also use the infix notation to run a target:
(e.g., nx serve myapp --configuration=production)

You can skip the use of Nx cache by using the --skip-nx-cache option.

Options:
      --version        Show version number                                                                                                                                                                                                                                                                                                                  [boolean]
      --prod           Use the production configuration                                                                                                                                                                                                                                                                                    [boolean] [default: false]
  -c, --configuration  Target configuration                                                                                                                                                                                                                                                                                                                  [string]
      --project        Target project                                                                                                                                                                                                                                                                                                                        [string]
      --output-style   Defines how Nx emits outputs tasks logs                                                                                                                                                                                                                                           [string] [choices: "dynamic", "static", "stream", "compact"]

Error: Passed 2 file paths to Git to hash, but received 0 hashes.

    at /Users/<censored>/repos/<censored>/node_modules/nx/src/hasher/git-hasher.js:38:19
    at Generator.next (<anonymous>)
    at fulfilled (/Users/<censored>/repos/<censored>/node_modules/tslib/tslib.js:115:62)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

@SynergyEvolved
Copy link

I think you have to run git hash-object and add the folder with the error to .gitignore.

@julianpoemp
Copy link

I think you have to run git hash-object and add the folder with the error to .gitignore.

thank you! I found out that the error happens as soon as I generate files outside the Nx scope (in my case a build from pkg) that's recognized by git. As soon as I add these files to .gitignore the problem is fixed.

@github-actions
Copy link

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants