New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to start or build with too many files #2354
Comments
Can graceful-fs be a solution? If yes, where to insert it in the app to globally patch the fs? |
I have the same issue. I am expected to have around 6000 pages. Right now the build fails with the same error as mentioned above. |
comanche2
added a commit
to comanche2/nuxt-content
that referenced
this issue
Feb 8, 2024
7 tasks
comanche2
added a commit
to comanche2/nuxt-content
that referenced
this issue
Feb 19, 2024
farnabaz
pushed a commit
that referenced
this issue
Feb 20, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Environment
Reproduction
I have 300 files (json), each around 300kb in size in the
content
folder. The app fails to start itself, i.e., onnpm run dev
, the app fatally fails with javascript heap memory errorSo, I spilt these 300 files into 40 files each, so now I have around 12000 files (json), each much smaller with 7-8 kb in size. Now, the app starts properly (i.e., no error on
npm run dev
. I'm able to query properly in dev, but the app fails fatally onnpm run build
.I'm using Nuxt 3 server app, i.e., Nitro Firebase preset. So, I'm using
serverQueryContent
in the/api/articles/index.ts
file to serve my response. In this file, I'mfind
ing all the 12000 json files, and then filtering them using normal filter function (because I want to query the files by the object keys of the same file, which is not possible with the module's query syntax). This is working properly in dev. On build, it fails.Describe the bug
First bug (few files, but large in size)
npm run dev
itself with error of memory heap.Second bug (many files, small in size)
On
npm run dev
, several red warnings abouttoo many files open
. But the app still starts properly.On
npm run build
, there's a fatal error related to unstorage thaterror [unstorage] Cannot stringify value!
. On inspection, I found that it is due toPromise.all
syntax in themap
in the content module's code. When I changed it to simplefor... of
loop, and added a wait time of 5 ms, this stringify error went away, and the app starts building for production. But then a new fatal error props up later related to tailwind config file that too many files open.Additional context
Can the module contributors kindly look into it? I want to add 160000 files ultimately, but the module fails at 12000 files itself. I tried removing the files, and found the module works fine (both during dev and build) with max 6000-7000 files of 7-8 kb.
Note all these files are json (object format).
Logs
No response
The text was updated successfully, but these errors were encountered: