Error: EPERM: operation not permitted, open

[SirWaddles]

I'm getting the above error on Windows while using electron-json-storage:4.1.2

I'm building an import procedure which does a large number of requests inside of a single Promise.all, and storage.get is being used inside of electron's session.webRequest.onBeforeSendHeaders (as an authentication mechanism)

I think this has to do with the concurrency in the event loop. There's a lot of calls on storage.get that stack up before getting resolved.

Only solution I can think of is that I implement a lazy cache somewhere, which I will probably end up doing. But I was wondering if this is a known limitation/issue of electron-json-storage.

Thanks,
Waddles

[jviotti]

Hi @SirWaddles ,

Thanks for the report! Do you have a whole stack trace I can take a look at? I'm basically trying to find out if this problem happens when locking or unlocking.

Is this happening on other OSes, or just Windows?

[valhalla13]

Hi, @jviotti my app is installed on more than 1k Windows and 400 Mac computers and 5-10% Windows users are facing the same issue. Everything is fine on the Mac.
I have noticed this error from last 3 weeks, earlier I was also thinking it only comes when locking unlocking the system. But it is not true. It comes anytime, and when the user tried to click ok it comes again continuously many times.

[jviotti]

@valhalla13 @SirWaddles Do you mean it comes on any fs function call? Does it also happens when locking and unlocking? Does it eventually succeed if you keep trying? Can you check if people that see this error have any anti virus installed? I remember seeing something similar with various anti virus which would request exclusive access to files while scanning them, and thus causing EPERM for anyone else trying to access them.

It'd really help if you can share a stack trace with me.

[SirWaddles]

@jviotti Hi, sorry about taking this long.

I don't think this is the trace you're after, but it seems to be the only one I'm able to print.

at storage.get (D:\project\electron\resources\index.js:163:25)
    at D:\project\electron\node_modules\electron-json-storage\lib\storage.js:181:16
    at D:\project\electron\node_modules\lockfile\lockfile.js:62:47
    at FSReqWrap.oncomplete (fs.js:135:15)

I don't have any anti-virus, apart from Microsoft Defender and this is happening in my environment.

[jviotti]

@SirWaddles Thanks a lot for the stack-trace; that helps a lot! Can you try v4.1.3 and see if the issue is gone? Windows Defender can definitely be the cause. It gave us a lot of head-aches on https://etcher.io :D

[zankooh]

I have the same exact problem with v4.1.3.
This might not be the problem, but I have noticed that the error occurs whenever I call storage.get(key) with different keys in the same script.

For example:
storage.get('item', function() {})
storage.get('anotherItem', function() {})

Now, I am not using these methods directly. I have made service wrappers and call functions which return promises that resolve the data returned from the storage.get() method. Even in this case, when I chain the promises (This is some badly written code. It is just an example.):

`promiseFunctionCall()

 .then(function() {

      anotherPromiseFunctionCall()

            .then(function() {

                    // do something

             })

 })`

[jviotti]

Hey @zankooh ,

Hm, that's very interesting. Do you have a stack trace you can share?

[ozooner]

I upgraded our app to 4.1.4 from 4.1.3 in hopes to get the bug fixed, but I can confirm that 4.1.4 still has the issue. I myself am unable to reproduce it, but it happens on about 5-10% of our windows users. There are very few OSX and Linux installs, so I can't rule them out completely, but no incidents there so far...
No stack to share unfortunately, as I can't reproduce it on my machine, but I can agree that it seems to happen when reading the file multiple times

[zankooh]

I think I managed to solve the problem. It might not the electron-json-storage itself. The problem might be file system related.

Whenever I read a .json file with electron-json-storage, the file gets temporarily locked (by the file system I assume?) and then it gets unlocked, being free to read without any problems. Now, if I try to read the same file while it's locked, I get the error. In order to avoid the problem, I store the data with electron-json-storage (for permanent storage) and localStorage at the same time, but read data from localStorage only. In case localStorage gets cleared, I read all data stored with electron-json-storage and write it to localStorage.

I haven't had the problem since.

[ozooner]

I solved it similarly:
On electron startup the program reads file into memory. Then all reads are done from memory, and writes are done by updating the storage file in memory and then calling a throttled function to write to file. If multiple writes happen fast, the throttle would just delay the writes to every 2 seconds and write most recent object from memory to the file.
https://codeburst.io/throttling-and-debouncing-in-javascript-b01cad5c8edf

[jviotti]

Very interesting! Presumably this is a Windows only issue, right? I think we could add logic to electron-json-storage to retry if it gets an EPERM during reads for this reason. I sadly only have access to a Windows VM nowadays, and I can't seem to be able to reproduce on that setup.

I might at least draft a patch and we can see how it goes.

[jviotti]

I just sent #122. Let me know how it goes!

[jviotti]

I published the possible fix as v4.1.5

[kevintravels93]

I have a similar issue. I can´t edit the config.json from the electron store. I get this error.

If I try to change the file in the explorer or if I try to change the file with the set method.

I manually changed the path inside the electron-store module. Now I can access the File and modify it.

My operating system is Windows 10

//const defaultCwd = (electron.app || electron.remote.app).getPath('userData');
const defaultCwd = 'C:/agentPaul/electronData/'

[jviotti]

Hey @kevintravels93

If I try to change the file in the explorer

Do you mean you can reproduce the EBUSY by trying to change the file with something like notepad.exe rather with the module?

I googled around and I found out that this tool allows you to inspect what processes are opening what files in Windows.

Can you revert back the directory setting, and once you reproduce the issue, try to see what processes and locking this file? See more instructions here: https://serverfault.com/a/1980.

This is supposedly the standard place in Windows to store application data, so sounds worth figuring out what in your system is causing problems there.