Add convenience method to read JSON file

[sindresorhus]

Is your feature request related to a problem? Please describe.

Reading a JSON file is a very common need. It would be nice to have a convenience method for it.

In addition to making it easier to do this common task, it could also:

• Strip UTF-8 BOM
• Make syntax errors more human-friendly

For example, I have a package for this which is quite popular and has 700 dependents, so it's clearly a need for it: https://www.npmjs.com/package/load-json-file

It is also quite a popular question on Stack Overflow: https://stackoverflow.com/questions/10011011/using-node-js-how-do-i-read-a-json-file-into-server-memory?rq=1

Having it in Node.js core would also help reduce dependency trees, which would in turn reduce npm install times and node_modules bloat. It would also improve DX by having better syntax errors.

Describe the solution you'd like

I propose adding a fs.readJson() and fs.readJsonSync() method.

Describe alternatives you've considered

Continue using https://www.npmjs.com/package/load-json-file

---

I didn't want this to dilute the proposal, but one future enhancement to this could be to make the JSON parsing fully async by doing it in a worker.

[targos]

I tried to do an initial implementation in #37944 but nobody approved it.

[aduh95]

If we land web methods like .text() and .arrayBuffer() to FileHandle like in #40021, I think we may also consider adding .json().
There's also the promise of being able to do await import('./pathToFile.json') if/when #37375 lands.

[jimmywarting]

with that said ☝️ then I don't think we will need any special fs.readJson function

[sindresorhus]

There's also the promise of being able to do await import('./pathToFile.json') if/when #37375 lands.

Importing something is semantically different from reading something directly from disk. I assume import caches the import. And bundlers will treat it differently, often inlining imports, but ignoring fs reading. In addition, fs methods read relative to CWD, while dynamic imports read relative to the file.

[sindresorhus]

with that said ☝️ then I don't think we will need any special fs.readJson function

FileHandle is not a replacement for fs methods.

[jasnell]

In the stream/consumers module that currently has not yet gone out in a release because it hasn't yet been backported to 16.x or 14.x (there's a small smever-major that it currently depends on)... there is a json() method that takes either a ReadableStream, stream.Readable or AsyncIterator as input...

so... you can do something like

await json(fs.createReadStream('...'));

And it should just work.

As a reminder, stream/consumers exposes:

• await arrayBuffer(input)
• await blob(input)
• await buffer(input)
• await text(input)
• await json(input)

Where input can be ReadableStream, stream.Readable, or AsyncIterator

This has already landed, just waiting to go out in a release.

[jimmywarting]

@jasnell, i experimented with stream consumer a bit... i think it could be useful if blob accepted a optional mime type as well (just a suggestion)
await blob(input, [type])

[jasnell]

@sindresorhus

I didn't want this to dilute the proposal, but one future enhancement to this could be to make the JSON parsing fully async by doing it in a worker.

You could do this fairly easily with FileHandle and the new stream/consumers... e.g. open the file as a FileHandle, transfer that to a Worker, use await json(fh.createWebStream()) and postMessage the results back. It would be fairly simple.

[jasnell]

@jimmywarting:

i experimented with stream consumer a bit... i think it could be useful if blob accepted a optional mime type as well (just a suggestion)

Yeah, let's take that to a separate issue. I think adding an options argument to each of the convenience consumers makes sense.

[benjamingr]

Can anyone explain what's wrong with requireing the JSON file? (or importing it in ESM land with json modules active)?

Like, why is fs.readJson('./foo.json better than require('./foo.json') (old) or await import('./foo.json')? Is the "moduleness" of it (caching behaviour etc) the problematic bit?

[sindresorhus]

@benjamingr #40089 (comment)

[benjamingr]

In addition, fs methods read relative to CWD, while dynamic imports read relative to the file.

Oh yeah that makes sense. Especially if this lets us (maybe?) parse the JSON on a libuv thread and not the main thread though I suspect the main use case for this is probably config files in which the pathing is the important bit.

[kaizhu256]

Can anyone explain what's wrong with requireing the JSON file? (or importing it in ESM land with json modules active)?

require reads the file once and caches it. I've come across use-cases where json-file is intentionally modified and needs to be physically re-read.

That said, typing JSON.parse(await fs.promises.readFile(...)) is not that much of a chore, nor does readability take that much of a hit.

[bmeck]

Probably the most important thing here is that files with a BOM do not parse with JSON.parse: JSON.parse('\uFFEF""')

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[sindresorhus]

Please keep this open.

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[github-actions]

There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[sindresorhus]

This is still a wanted feature. Can someone please reopen?

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[sindresorhus]

Please keep this open.

[bnoordhuis]

I've read through the discussion but it's unclear to me if a) there is consensus whether it should be added, and b) what the API should look like, assuming the answer to A is "yes." (And if it's "no", then I suggest we close this out.)

[targos]

As reported in #40089 (comment), there was unfortunately no collaborator support for just adding convenience methods.

[bnoordhuis]

Okay, then I'll close it. Thanks anyway for the suggestion, @sindresorhus.