Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating a new JSON file should not throw error when empty (or crash) #2299

Closed
andy-bond opened this issue Apr 1, 2022 · 5 comments
Closed

Comments

@andy-bond
Copy link

Describe the bug
While the 11ty-dev-server is running, if you create a new JSON file in the _data directory, the server will crash with the following error.

[11ty] Watching…
[11ty] File added: src/_data/test.json
[11ty] Eleventy fatal watch error: (more in DEBUG output)
[11ty] /Users/bondandy/Projects/11ty/design-system/src/_data/test.json: Unexpected end of JSON input (via SyntaxError)
[11ty] 
[11ty] Original error stack trace: SyntaxError: /Users/bondandy/Projects/11ty/design-system/src/_data/test.json: Unexpected end of JSON input
[11ty]     at parse (<anonymous>)
[11ty]     at Object.Module._extensions..json (node:internal/modules/cjs/loader:1167:22)
[11ty]     at Module.load (node:internal/modules/cjs/loader:981:32)
[11ty]     at Function.Module._load (node:internal/modules/cjs/loader:822:12)
[11ty]     at Module.require (node:internal/modules/cjs/loader:1005:19)
[11ty]     at require (node:internal/modules/cjs/helpers:102:18)
[11ty]     at TemplateData.getDataValue (/Users/bondandy/Projects/11ty/design-system/node_modules/@11ty/eleventy/src/TemplateData.js:483:25)
[11ty]     at TemplateData.getAllGlobalData (/Users/bondandy/Projects/11ty/design-system/node_modules/@11ty/eleventy/src/TemplateData.js:273:29)
[11ty]     at async TemplateData.getData (/Users/bondandy/Projects/11ty/design-system/node_modules/@11ty/eleventy/src/TemplateData.js:332:24)
[11ty]     at async Eleventy.restart (/Users/bondandy/Projects/11ty/design-system/node_modules/@11ty/eleventy/src/Eleventy.js:270:5)
ERROR: "watch:eleventy" exited with 1.
The terminal process "/bin/zsh '-c', 'npm run start'" terminated with exit code: 1.

If you create a JSON file in a different directory, say a file called docs.json in a directory called docs (to add tags to all files in folder) you will receive the same error but will be able to continue. That being said, I would rather it not show an error at all and simply ignore empty JSON files (error if they are malformed, sure).

[11ty] Watching…
[11ty] File added: src/docs/docs.json
[11ty] Problem writing Eleventy templates: (more in DEBUG output)
[11ty] /Users/bondandy/Projects/11ty/design-system/src/docs/docs.json: Unexpected end of JSON input (via SyntaxError)
[11ty] 
[11ty] Original error stack trace: SyntaxError: /Users/bondandy/Projects/11ty/design-system/src/docs/docs.json: Unexpected end of JSON input
[11ty]     at parse (<anonymous>)
[11ty]     at Object.Module._extensions..json (node:internal/modules/cjs/loader:1167:22)
[11ty]     at Module.load (node:internal/modules/cjs/loader:981:32)
[11ty]     at Function.Module._load (node:internal/modules/cjs/loader:822:12)
[11ty]     at Module.require (node:internal/modules/cjs/loader:1005:19)
[11ty]     at require (node:internal/modules/cjs/helpers:102:18)
[11ty]     at TemplateData.getDataValue (/Users/bondandy/Projects/11ty/design-system/node_modules/@11ty/eleventy/src/TemplateData.js:483:25)
[11ty]     at TemplateData.combineLocalData (/Users/bondandy/Projects/11ty/design-system/node_modules/@11ty/eleventy/src/TemplateData.js:362:36)
[11ty]     at TemplateData.getTemplateDirectoryData (/Users/bondandy/Projects/11ty/design-system/node_modules/@11ty/eleventy/src/TemplateData.js:373:37)
[11ty]     at processTicksAndRejections (node:internal/process/task_queues:96:5)
[11ty] Wrote 0 files in 0.01 seconds (v2.0.0-canary.4)

To Reproduce
Steps to reproduce the behavior:

  1. Using 11ty 2.0.0-canary.4, run server with eleventy --serve
  2. Create a new file in the data directory _data/test.json
  3. Crash
  4. Repeat in a different folder and see error but no crash

Expected behavior
No error for empty JSON file - error for malformed JSON (ideally no crashing)

Environment:

  • OS and Version: Mac, MacOS 12.3
  • Eleventy Version: 2.0.0-canary-4
@pdehaan
Copy link
Contributor

pdehaan commented Apr 1, 2022

But isn't an empty JSON file technically malformed JSON?

I ran some quick tests using require(), fs-extra.readJsonSync(), and json5 and they all seemed to throw errors if I tried to read an empty JSON file (ie: touch empty.json).

require("json5/lib/register");

const fse = require("fs-extra");

try {
  // SyntaxError: /private/tmp/json-test/empty.json: Unexpected end of JSON input
  const emptyRequire = require("./empty.json");
  console.log({emptyRequire});
} catch (err) {
  console.error("1.", err.message);
}

try {
  // SyntaxError: ./empty.json: Unexpected end of JSON input
  const emptyFse = fse.readJsonSync("./empty.json");
  console.log(emptyFse);
} catch (err) {
  console.error("2.", err.message);
}

try {
  // SyntaxError: /private/tmp/json-test/empty.json5: JSON5: invalid end of input at 1:1
  const emptyJson5 = require("./empty.json5");
  console.log({emptyJson5});
} catch (err) {
  console.error("3.", err.message);
}

try {
  const emptyJson = safeLoad("./empty.json");
  // { emptyJson: {} }
  console.log({emptyJson});
} catch (err) {
  console.error("4.", err.message);
}

function safeLoad(p) {
  try {
    return require(p);
  } catch (err) {
    return {};
  }
}
1. /private/tmp/json-test/empty.json: Unexpected end of JSON input
2. ./empty.json: Unexpected end of JSON input
3. /private/tmp/json-test/empty.json5: JSON5: invalid end of input at 1:1
{ emptyJson: {} }

I'm not too familiar w/ the internals of 11ty's global data files vs directory data files, but it sounds like directory data files might have a slightly better developer experience. Wrapping the JSON loading in a try..catch and returning an empty object on error might be safer.

@pdehaan
Copy link
Contributor

pdehaan commented Apr 1, 2022

Baby steps...
Might have a solution, but it isn't super great.

const json5 = require("json5");

module.exports = eleventyConfig => {
  eleventyConfig.addDataExtension("json5", contents => json5.parse(contents));
  ...
};

It seems if I use json5 as a custom data file format (also see https://www.npmjs.com/package/json5 for details). It seems to handle empty files a bit better when I use json5.parse(), plus you can add comments to your JSON files, unquoted keys, etc…
But the downside is that you'd have to rename your data files to .json5 (or whatever). I tried overriding the .json extension, but that didn't work out for me; see #2300.

Anyways, not sure if it helps or not, but I learned something today, so it was a good day.

@andy-bond
Copy link
Author

I totally understand if it this isn't something that is fixable - just thought it might trip others up the first time they create a global data file.

I'm not familiar with 11ty internals, but before trying to read as JSON, could you first use fse.readFileSync if the filetype is .json and then check for empty / no content? If there is content, proceed with JSON processing however it normally works. If there is no content, throw warning and skip loading the file as JSON?

Definitely adds a step when processing which might not be ideal though. Thanks for investigating regardless!

@zachleat zachleat added enhancement needs-votes A feature request on the backlog that needs upvotes or downvotes. Remove this label when resolved. and removed needs-triage labels Apr 11, 2022
@zachleat
Copy link
Member

I agree that this change would be a DX improvement! 👍🏻 I think it makes the most sense to add this to the enhancement queue

This repository is now using lodash style issue management for enhancements. This means enhancement issues will now be closed instead of leaving them open.

View the enhancement backlog here. Don’t forget to upvote the top comment with 👍!

@zachleat zachleat removed the needs-votes A feature request on the backlog that needs upvotes or downvotes. Remove this label when resolved. label Dec 16, 2022
@zachleat zachleat added this to the Eleventy 2.0.0 milestone Dec 16, 2022
@zachleat
Copy link
Member

This is shipping with 2.0.0-canary.19, thank you!

Fixed by 7c03471

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants