RangeError: Maximum call stack size exceeded

[oorestisime]

I am getting this error when trying to deploy a large bucket.

RangeError: Maximum call stack size exceeded
    at /home/circleci/bold/frontend/node_modules/gatsby-plugin-s3/node_modules/klaw/src/index.js:53:23
    at go$readdir$cb (/home/circleci/bold/frontend/node_modules/graceful-fs/graceful-fs.js:162:14)
    at FSReqWrap.args [as oncomplete] (fs.js:140:20)

Seems to be caused by jprichardson/node-klaw#37

Not sure if anyone has workaround.
I am willing to make the change to walk if you agree , or if there's something else to do here to fix it :)

[YoshiWalsh]

I don't think we're particularly attached to klaw. I'd be happy to review a PR that replaces it with walk.

[oorestisime]

Hey i am wondering if you have any ideas on how to test this out. Would it work if i build the plugin and just call the bin.js with the right environment variables?
if this works i could test on circleci as well by sshing in to test it works with our large bucket.

[YoshiWalsh]

You can use npm link to symlink your local copy of the plugin into your site. See here: https://www.gatsbyjs.com/docs/creating-a-local-plugin/#using-npm-link-or-yarn-link

This is likely something that we won't test in our e2e test suite, as I think generating and uploading ~200,000 files each test run would make the tests quite slow.

Thanks!

[oorestisime]

Looks like its way too slow walk. plus it needed a lot of changes given that it doesn't have streamable reads. i making a PR in klaw to fix. Should i keep open to notify once its released so we can update the plugin dependency as well?

[YoshiWalsh]

Please do!