Error: EPERM: operation not permitted

[Grzegorzsa]

I've got some errors constantly reported by express server. Sessions seem to be working in spite of the error.

Windows 10/64
npm ver 6.1.0
node ver 10.5.0

{ [Error: EPERM: operation not permitted, rename 'V:\www\docs-viewer\sessions\4vNjE4HUhbBsrMUwtRdksqIaoFkHIxkE.json.4225269665' -> 'V:\www\docs-viewer\sessions\4vNjE4HUhbBsrMUwtRdksqIaoFkHIxkE.json']
errno: -4048,
code: 'EPERM',
syscall: 'rename',
path:
'V:\www\docs-viewer\sessions\4vNjE4HUhbBsrMUwtRdksqIaoFkHIxkE.json.4225269665',
dest:
'V:\www\docs-viewer\sessions\4vNjE4HUhbBsrMUwtRdksqIaoFkHIxkE.json' }

[valery-barysok]

looks like you have problem with permissions in this folder or may be it is related to simultaneous rename of files to the same file. In second case it is like optimistic lock conflict because you try to save stale session state.

[Grzegorzsa]

There are no special permission set to this folder and files are created by your plugin. I don't do anything special with those files. The plugin was installed and configured that's all. The output comes from function that catches unhandled exceptions. When I disable your plugin errors stops showing up. Apart from throwing errors it seems to be working fine.

[Grzegorzsa]

I've found somenthing like this: isaacs/node-graceful-fs#104
Maybe somehow it is related to this issue

[acritelli]

Same issue here. Nothing special about the permissions on the directory: it's created automatically by the module.

Windows 10/64
NPM: 5.6.0
Node: 8.11.1

The error will intermittently show up and, when it does, it causes transient issues with those sessions. For example: a user may not be able to log in.

[ksfreitas]

Same here on Windows. This ocurrs due multiple session access (when cache is off, for example, this session is called for every resource like css, images, etc).

Each session access try to save the lastAccess property. Due some concurrency problem, only one will save correctly.

[ksfreitas]

The problem is located in write-file-atomic library. Related issue: npm/write-file-atomic#28

[iarna]

Hey, I just did a release yesterday with a bunch of fixes that may ease this issue a bit, specifically, it was previously leaking filehandles which may have made this worse. Can I get y'all to try updating to write-file-atomic@2.4.2 and see if that helps?

[valery-barysok]

@iarna, thanks. I will try to update dependency.

[Grzegorzsa]

Hi, after updating write-file-atomic those errors stopped showing up :)
Unfortunately I've got bunch of other errors

1) When I log out:
[session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'V:\www\docs-viewer\sessions\fz5sqDbJzayaXJAlUPcbHv_sygN5UU6x.json'
session: Session {
cookie:
{ path: '/',
_expires: 2022-03-28T20:05:10.487Z,
originalMaxAge: 99999999999,
httpOnly: true,
secure: false } }
GET /user/login 200 61.160 ms - 2005
[session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'V:\www\docs-viewer\sessions\fz5sqDbJzayaXJAlUPcbHv_sygN5UU6x.json'
[session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'V:\www\docs-viewer\sessions\fz5sqDbJzayaXJAlUPcbHv_sygN5UU6x.json'
[session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'V:\www\docs-viewer\sessions\fz5sqDbJzayaXJAlUPcbHv_sygN5UU6x.json'

2) When I try to log-in again - first attempt failed with errors:
[session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'V:\www\docs-viewer\sessions\R85sM_bOfyxEYGhzPK8pTGi7fuGh2ktN.json'
[session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'V:\www\docs-viewer\sessions\Q03iNngArQOWpFhb0zufBrb38cyW0q9c.json'
GET / 302 53.701 ms - 66
[session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'V:\www\docs-viewer\sessions\Q03iNngArQOWpFhb0zufBrb38cyW0q9c.json'
session: Session {
cookie:
{ path: '/',
_expires: 2022-03-28T20:08:10.969Z,
originalMaxAge: 99999999999,
httpOnly: true,
secure: false } }

3) Second attempt to log-in is successful - no errors

So, when I am logged in no errors are showing. When I log out I've got errors and I have to log in twice as first attempt fails

[jonathangaldino]

I'm having this issue as well. Does anyone have a fix?

[migueldaipre]

I'm having same issue. Does anyone have a fix ?

[davalapar]

Looked into this a bit, I'm quite certain it's write-file-atomic causing this.

I've encountered same range of errors in one of my projects on moderate to heavy amount of calls involving opening (and closing) of file descriptors, and renaming the files themselves.

Better approach is to literally just keep the file descriptors open during writes (on both files, asynchronously) then close them synchronously once done. No need for fancy file renames, since writes on files a and b can be done asynchronous-yet-one-after-another way.

[kdemarest]

I'm getting the same problem:
[session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open ...
It started after I upgraded from a prior version of session-file-store.

[jas-]

In all cases have any of the following been inspected?

1. Deleted files in use?
2. Does the ENOENT files actually exist and correlate to the requested session?
3. Upon upgrade has the existing server sessions been cleared or were they left to resume?

There are several issues regarding Windows 10, anti ransomware protections and NodeJS versions <10 that relate to this error.

[kbjwes77]

I'm still getting this error. Running on latest version of node.js on an AWS linux VM.

[session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open 'sessions/asdjasdaodhhfiqwf.json'

All dependencies are up to date. Any ideas why this could still be happening?

[Adepusrikanth9666]

C:>browser-sync start --server --directory --files "**/*"
events.js:287
throw er; // Unhandled 'error' event
^

Error: EPERM: operation not permitted, watch 'C:\Documents and Settings'
at FSWatcher.start (internal/fs/watchers.js:169:26)
at Object.watch (fs.js:1339:11)
at createFsWatchInstance (C:\Users\sainaveen\AppData\Roaming\npm\node_modules\browser-sync\node_modules\chokidar\lib\nodefs-handler.js:38:15)
at setFsWatchListener (C:\Users\sainaveen\AppData\Roaming\npm\node_modules\browser-sync\node_modules\chokidar\lib\nodefs-handler.js:81:15)
at FSWatcher.NodeFsHandler._watchWithNodeFs (C:\Users\sainaveen\AppData\Roaming\npm\node_modules\browser-sync\node_modules\chokidar\lib\nodefs-handler.js:233:14)
at FSWatcher.NodeFsHandler._handleDir (C:\Users\sainaveen\AppData\Roaming\npm\node_modules\browser-sync\node_modules\chokidar\lib\nodefs-handler.js:429:19)
at FSWatcher. (C:\Users\sainaveen\AppData\Roaming\npm\node_modules\browser-sync\node_modules\chokidar\lib\nodefs-handler.js:477:19)
at FSWatcher. (C:\Users\sainaveen\AppData\Roaming\npm\node_modules\browser-sync\node_modules\chokidar\lib\nodefs-handler.js:482:16)
at FSReqCallback.oncomplete (fs.js:168:5)
Emitted 'error' event on FSWatcher instance at:
at FSWatcher._handleError (C:\Users\sainaveen\AppData\Roaming\npm\node_modules\browser-sync\node_modules\chokidar\index.js:260:10)
at createFsWatchInstance (C:\Users\sainaveen\AppData\Roaming\npm\node_modules\browser-sync\node_modules\chokidar\lib\nodefs-handler.js:40:5)
at setFsWatchListener (C:\Users\sainaveen\AppData\Roaming\npm\node_modules\browser-sync\node_modules\chokidar\lib\nodefs-handler.js:81:15)
[... lines matching original stack trace ...]
at FSReqCallback.oncomplete (fs.js:168:5) {
errno: -4048,
syscall: 'watch',
code: 'EPERM',
path: 'C:\Documents and Settings',
filename: 'C:\Documents and Settings'
}

i have an error while isstalling the browser sync any one can help pz?

[dillu24]

Currently I have the same issue Error: EPERM: operation not permitted, rename

Is there a fix planned for this?

[valery-barysok]

Can you figure out with permissions on your system and check that issue is not there and path is correct? Try with some other short path with right permissions without whitespaces ? and provide full details about your issue context, please.

[dillu24]

Can you figure out with permissions on your system and check that issue is not there and path is correct? Try with some other short path with right permissions without whitespaces ? and provide full details about your issue context, please.

The path and permissions are correct, as the files are correctly outputted in the desired folder.

[alexb-uk]

This appears to be a Windows Security Virus & Threat Protection issue.

Disabling "Real-time protection" removes the error. Some people appear to have had luck adding the folder to the exclusion list but this did not work for me.

Since we only use Windows on dev environments I will probably just live with the error message since it doesn't appear to effect session storage.

[aklietz]

The problem is that when a Windows AV software package spots a new file being created it likes to quietly map it for snooping (CreateFileMapping), and if that newborn file is also being renamed by another process immediately after creation (e.g, to move-replace another file) node's MoveFileEx can fail with a transient ERROR_ACCESS_DENIED (aka EPERM). I've used Process Monitor to spot Microsoft's own MsMpEng.exe in Windows Defender doing this in my sessions folder several times a day.

session-file-store is still using an old version of write-file-atomic from 2017 (1.3.1), which I suspect does not implement the necessary backoff-retry mechanism correctly (or maybe the fault is in its underlying transitive dependency on an old version of graceful-fs; the facts are confusing to me).

The current version of write-file-atomic is 3.0.3. It has dropped the dependency on graceful-fs. I tested it on my devel server against session-file-store with npm-force-resolutions and it seems to work wonderfully.

The main reason for the breaking change was their dropping of support for node 6. If session-file-store bumped a major version with a change log notice about also dropping support for node 6 it might be able to use it as-is. (Obviously it will need a lot of testing first.)

[valery-barysok]

@aklietz, will try to update dependencies soon.

[aklietz]

@valery-barysok, thanks. I hit the EPERM problem again even with write-file-atomic bumped up to 3.0.3, although it is happening a lot more rarely now. Taking out graceful-fs seems to help but it's not a complete fix.

I had assumed that node 12.18.3 implemented the exponential fallback heuristic for fs.rename() to retry MoveFileEx several times before giving up. However, a cursory inspection of the source shows this does not appear to be the case.

I looked at graceful-fs again, and while it does appear to be doing the exponential fallback heuristic for certain file operations, fs.rename() does not appear to be one of them.

Microsoft does provide a method of calling NtSetInformationFile with FILE_RENAME_INFORMATION that appears to do an atomic rename on the same volume.

[KEMBL]

How to update session-file-store deep dependency write-file-atomic version using resolutions in package.json

a) add
"resolutions": { "session-file-store/**/write-file-atomic": "^3.0.3" },

b) start yarn install

c) check yarn list write-file-atomic

session-file-store works fine, but still, Error: EPERM: operation not permitted, rename errors appearing despite the fact realtime-protection is off

[sakarit-zz]

@valery-barysok Please release the newest version (1.5.0) to npm!

[valery-barysok]

@sakarit, i do know how i missed that, thank you. Done.

[JoshMayberry]

I am still seeing this issue, even though write-file-atomic is on version 3.0.3 and session-file-store is on version 1.5.0.

[aklietz]

@valery-barysok, thanks. I hit the EPERM problem again even with write-file-atomic bumped up to 3.0.3, although it is happening a lot more rarely now. Taking out graceful-fs seems to help but it's not a complete fix.

I had assumed that node 12.18.3 implemented the exponential fallback heuristic for fs.rename() to retry MoveFileEx several times before giving up. However, a cursory inspection of the source shows this does not appear to be the case.

I looked at graceful-fs again, and while it does appear to be doing the exponential fallback heuristic for certain file operations, fs.rename() does not appear to be one of them.

Microsoft does provide a method of calling NtSetInformationFile with FILE_RENAME_INFORMATION that appears to do an atomic rename on the same volume.

@JoshMayberry, @valery-barysok The proper way to resolve this issue is to fix Node so that fs.rename() does an atomic rename using NtSetInformationFile with the argument FILE_RENAME_INFORMATION and the flags FILE_RENAME_REPLACE_IF_EXISTS | FILE_RENAME_POSIX_SEMANTICS instead of calling MoveFileEx.

[aklietz]

@JoshMayberry, @valery-barysok I checked the latest NodeJS (16.9.1) and confirmed that fs.rename still wrongly calls MoveFileEx when it ought to be calling NtSetInformationFile with the argument FILE_RENAME_INFORMATION with the flags FILE_RENAME_REPLACE_IF_EXISTS | FILE_RENAME_POSIX_SEMANTICS.

I suspect that the failure to request FILE_RENAME_POSIX_SEMANTICS is the root of the problem. fs.rename assumes that rename() is atomic per the POSIX (Linux) standard but is failing to ask Windows to provide said semantics.

[Ludovicmanga]

Hi, I use node Js, with passport and express-session.
I kept blocked for more than a day in it until I realize that, due to a design error in my react frontend, I was just sending way too many requests to my backend at a specific moment, in a very short time (like a hundred requests in a few seconds). Making sure that I don't send these tons of requests solved it for me.

I could not explain why the fact there is too many requests blocked the access though