Inconsistent reporting of offline storage quota

[mann-david]

Expected Behavior
According to all documentation I can find, including Issue #4550, Electron should have 1/3 of free space available for offline storage. In addition, our apps should only be able to use 20% of that, per https://developer.chrome.com/apps/offline_storage#table.

Actual behavior
I run the following code inside my Electron app:

const estimation = await navigator.storage.estimate();
console.log(`Quota: ${estimation.quota}`);
console.log(`Usage: ${estimation.usage}`);

and it reports that I actually have 100% of my free disk space available (i.e my 200GB drive has 105 GB free, and the code reports 105 GB available for use). I've tried this on multiple machines with 3 different Electron apps and it is consistent.

If it matters, I'm specifically interested in IndexedDb.

To Reproduce
Run the above code inside any Electron app (either in the app or just in the console inside the renderer process). Compare the results to the actual free space on your drive.

Additional Information
Primarily, I'm just looking for a clarification on whether what I'm seeing in the code is correct, that we have 100% of free disk space, or whether the documentation is correct and the code incorrect. Secondarily, I'd like to know if Electron will ever evict my data (specifically IndexedDb).

Thanks.

[welcome]

👋 Thanks for opening your first issue here! If you're reporting a 🐞 bug, please make sure you include steps to reproduce it. We get a lot of issues on this repo, so please be patient and we will get back to you as soon as we can.

To help make it easier for us to investigate your issue, please follow the contributing guidelines.

[BitPhoenix]

Interested in the answer to this as well

[mann-david]

Tested again in Electron 4.0.5 with the same results. Is there any official word?

[jacobq]

I wonder if this has anything to do with the disabling of the DOM storage quota checks in dom_storage_limits.patch.
(#13465)

[jacobq]

According to #9498 (comment) it sounds like there is "unlimited and durable storage for all origins", which seems like it would agree with the "100% available" result you mentioned.

[mann-david]

@jacobq: agree, but it would be nice to see this officially documented somewhere. I can't base my app on this assumption and then have data loss because something got evicted.

[r10-dev]

Has anyone found the documentation for the limits yet?

[lvkins]

@elrDevelopment

https://en.wikipedia.org/wiki/Web_storage

However, Electron has a patch for dom storage limits: https://github.com/electron/electron/blob/master/patches/chromium/dom_storage_limits.patch

[jacobq]

However, Electron has a patch for dom storage limits: https://github.com/electron/electron/blob/master/patches/chromium/dom_storage_limits.patch

@elrDevelopment I believe that's only for localStorage and sessionStorage (and in current electron 9 nightlies the limit for those is 100MiB), not IndexedDB. So far I have not seen it officially documented, but I believe there is no limit for storage in IndexedDB

[r10-dev]

I believe that to be the case as well. But cant find anything in the DOCS about it.

[lvkins]

This is implementation-specific I believe. The docs regarding storage rules can be found here, so the actual limit can range up to 2GB

[mann-david]

@lvkins: Thanks, but the contents on the mozilla site and wikipedia are pretty generic, and not really what this issue is about. The original question is specifically about IndexedDb. The code shown above contradicts everything I can find documented regarding IndexedDb limits and data eviction for Chrome. I'm specifically trying to determine what the situation is for Electron which does not appear to be documented anywhere.

I've tested again in Electron 7.1.2 and still see the same results: available storage quota is the total free space on the drive, minus any space already used for persistent storage.

[lvkins]

Indeed, those links I posted were directed to the @elrDevelopment that asked for the docs.

Regarding this issue, that I'm subscribing for few months out of curiosity, A good idea might be to test chromium behavior on that, because there seem to be no regulations in electron regarding this.

If you will be able to reproduce same behavior on standalone chromium build and you are determined enough, you can dig into the chromium IndexedDb implementation. There is a documentation available if you're interested. Quite outdated, but might be a good starting point for digging.

Otherwise, if you don't wanna get your hands dirty, I encourage you to post this issue at chromium repo.

[r10-dev]

@lvkins: Thanks, but the contents on the mozilla site and wikipedia are pretty generic, and not really what this issue is about. The original question is specifically about IndexedDb. The code shown above contradicts everything I can find documented regarding IndexedDb limits and data eviction for Chrome. I'm specifically trying to determine what the situation is for Electron which does not appear to be documented anywhere.

I've tested again in Electron 7.1.2 and still see the same results: available storage quota is the total free space on the drive, minus any space already used for persistent storage.

@mann-david not sure why this is so hard for a person from the electron team couldn't answer this.

[jacobq]

@zcbenz @deepak1556 Would you mind taking a moment to help clear this up? There seem to be some apparently contradictory claims here:

1. Limit is same as Chromium (Is that still 1/3? A quick test in my browser tells me probably not.)
   Question: IndexedDB size limit #4550 (comment)
   IndexedDb gets corrupted on Electron 7 #21558 (comment)
2. Unlimited
   No storage quota provided in QuotaSettings #9498 (comment)

• Does electron change the way that Chromium applies these quotas? (I know there's a patch for DOM storage, but I didn't see anything obviously related to IndexedDB)
• If not, is there anything being done to normalize this across versions or does each version just stand on its own?

[mann-david]

I've given up on IndexedDB simply because I couldn't get an answer on this and couldn't risk my product on it changing.

[zcbenz]

It seems that the storage quota in Electron is unlimited:
https://github.com/electron/electron/blob/master/shell/browser/special_storage_policy.cc#L20-L22

I think we can provide an API to let users set the storage quota themselves.

[zcbenz]

I'm looking into how Chromium calculates the storage quota, it seems that their algorithm is:
min(TotalDiskSpace * 0.8, FreeDiskSpace), so for most people all free disk space would be returned.

I have also tested with Chrome 79 and it returned all free disk space as storage quota.

[zcbenz]

This behavior seems to be intended:
https://bugs.chromium.org/p/chromium/issues/detail?id=897276

[ccorcos]

I did some testing myself and it appears that indexeddb quota is pretty high, if not unlimited as stated.

https://github.com/ccorcos/electron-indexeddb-quota

[zcbenz]

I'm closing this issue since Electron's behavior aligns with Chromium.