Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(core): gracefully handle errors when writing the latest output hash cache files #6959

Merged
merged 2 commits into from Oct 1, 2021

Conversation

minijus
Copy link
Contributor

@minijus minijus commented Sep 8, 2021

Instead of failing the task run process it will allow the process to continue.

Fixes of #6957

@vercel
Copy link

vercel bot commented Sep 8, 2021

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/nrwl/nx-dev/7s44TAfKXvMDhPwwPDinPQzpDsR3
✅ Preview: Canceled

[Deployment for f6b611e canceled]

@leosvelperez
Copy link
Member

@minijus thanks for contributing!

A try..catch there definitely wouldn't hurt, but before committing that change, I'd like to understand better when or why this issue happens. I saw you mentioned in this comment that it's only happening on lint executions, is that so? If that's the case, are you able to come up with a scenario where I can reproduce it?

@minijus
Copy link
Contributor Author

minijus commented Sep 23, 2021

@leosvelperez Nx local cache solution is written in a way that it never considers failures of atomic fs operations. Usually, that never happens - especially on single dev machine.

However, using local cache on CI where cache directory is mounted is another story. In our case we mount NFS based cache volume to CI builds. There are many parallel reads/writes to that directory and sometimes they fail. Ideally cache implementation should be written in a way that failure to read/write cache would not impact the run itself, instead it would fallback to running actual task.

@leosvelperez
Copy link
Member

@minijus I agree, I didn't say otherwise :)

Applying this fix is not in doubt, but we might be hiding another issue looking at your comment here where you say It does seem that failures on cache read are happening only on lint tasks. That's what I want to understand. Could you please clarify if this is only happening when linting and how could I reproduce it?

@minijus
Copy link
Contributor Author

minijus commented Sep 23, 2021

We have started seeing this failures only on 12.9 (there were no failures on 12.8). The difference is that 12.9 adds "outputs": ["{options.outputFile}"]to lint task definition in angular.json. I am not sure if that is the root cause, but it could be related.

@leosvelperez leosvelperez changed the title fix(core): wrap writing cache hash into try/catch fix(core): gracefully handle errors when writing the latest output hash cache files Oct 1, 2021
@leosvelperez leosvelperez enabled auto-merge (squash) October 1, 2021 13:28
Copy link
Member

@leosvelperez leosvelperez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing!

minijus and others added 2 commits October 1, 2021 16:18
Instead of failing the task run process it will allow the process to continue.

Fixes of #6957
@github-actions
Copy link

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants