Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: parse yaml with custom nom-based parser written in rust #4807

Draft
wants to merge 94 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
7a01f1c
feat!: parse yaml with custom nom-based parser written in rust
paul-soporan Sep 1, 2022
8fa276c
chore: dedupe
paul-soporan Sep 2, 2022
981cc44
fix: fix parseSyml return type
paul-soporan Sep 2, 2022
73ffe86
fix: use any
paul-soporan Sep 2, 2022
74e8327
build: enable more verbose logging
paul-soporan Sep 2, 2022
728167a
refactor: extract input type to make it easier to change it
paul-soporan Sep 3, 2022
2c06b23
perf: make parser work on &[u8] instead of &str
paul-soporan Sep 3, 2022
0c64066
Merge remote-tracking branch 'origin/master' into paul/feat/yaml-pars…
paul-soporan Sep 3, 2022
3baf40c
fix: use fold_many1 because fold_many0 swallows errors
paul-soporan Sep 4, 2022
759b0f3
feat: better errors via nom-supreme
paul-soporan Sep 4, 2022
f6adeb2
refactor: remove duplicate import
paul-soporan Sep 4, 2022
82b96bc
feat: implement support for flow sequences
paul-soporan Sep 4, 2022
478d75a
feat: allow any expression at the top level
paul-soporan Sep 7, 2022
87b9174
test: add a few basic tests
paul-soporan Sep 7, 2022
2aaf99e
feat: implement support for empty double quoted scalars
paul-soporan Sep 7, 2022
8fcae78
feat: implement support for single-quoted scalars
paul-soporan Sep 8, 2022
e50e542
feat: implement support for empty single quoted scalars
paul-soporan Sep 8, 2022
fed02e8
feat: implement support for flow mappings
paul-soporan Sep 9, 2022
54d3ed0
feat: allow line endings in flow mappings
paul-soporan Sep 9, 2022
c019213
feat: allow line endings in flow sequences
paul-soporan Sep 9, 2022
1f42a37
feat: allow flow collections to contain each other
paul-soporan Sep 9, 2022
3b8b874
feat: allow trailing commas inside flow sequences
paul-soporan Sep 9, 2022
abaf066
test: add some tests for block nodes
paul-soporan Sep 9, 2022
fce56ce
feat: allow flow collections to contain compact mappings
paul-soporan Sep 9, 2022
5feef5f
perf: don't use the json! macro
paul-soporan Sep 9, 2022
366bd35
refactor: don't use unnecessary to_owned
paul-soporan Sep 9, 2022
7439c57
refactor: reuse `flow_expression`
paul-soporan Sep 12, 2022
83fe883
Merge branch 'master' into paul/feat/yaml-parser-via-nom
paul-soporan Sep 29, 2022
7ba34ee
style: fix comment
paul-soporan Sep 29, 2022
7df7698
feat: throw on duplicate mapping entries
paul-soporan Sep 29, 2022
1ce97ff
refactor: tweaks
paul-soporan Sep 29, 2022
5ab6782
feat: support `overwriteDuplicates`
paul-soporan Sep 30, 2022
18bdbcf
fix: pass `overwriteDuplicates` when parsing a lockfile with merge co…
paul-soporan Sep 30, 2022
2db65d9
refactor: `overwriteDuplicates` -> `overwriteDuplicateEntries`
paul-soporan Sep 30, 2022
bb0d6e8
refactor: use `parse_separated_terminated`
paul-soporan Sep 30, 2022
4929063
fix: don't panic on duplicate key
paul-soporan Sep 30, 2022
8d7c859
refactor: reorder expression parsers
paul-soporan Oct 1, 2022
d382372
chore: upgrade wasm-bindgen
paul-soporan Oct 1, 2022
55bc248
chore: upgrade rust toolchain
paul-soporan Oct 1, 2022
3962cf6
test: move "it should merge duplicates" test
paul-soporan Oct 2, 2022
b12c010
refactor: use `map_locations`
paul-soporan Oct 2, 2022
b5f4cde
chore: upgrade rust toolchain
paul-soporan Oct 12, 2022
8c3ef77
refactor: extract block expressions to `block_expression`
paul-soporan Oct 12, 2022
6d0884d
refactor: tweaks
paul-soporan Oct 12, 2022
b0a0db1
refactor: more tweaks
paul-soporan Oct 15, 2022
e55d7e7
feat: support block expressions inside block sequences
paul-soporan Oct 15, 2022
f3926ce
fix: make sure source ends with `\n`
paul-soporan Oct 15, 2022
763740e
refactor: simpler solution for block expressions in block sequences
paul-soporan Oct 15, 2022
4045cb1
refactor: more tweaks
paul-soporan Oct 15, 2022
5e88dcc
chore: update deps
paul-soporan Oct 15, 2022
1bbf171
fix: don't show implementation details in the error stack
paul-soporan Oct 16, 2022
1ba11ba
feat: support legacy lockfiles
paul-soporan Oct 16, 2022
d0bfb09
feat: support escaping single quotes in single quoted scalars
paul-soporan Oct 16, 2022
cc60885
Merge branch 'master' into paul/feat/yaml-parser-via-nom
paul-soporan Oct 25, 2022
601954e
test: fix `LEGACY_REGEXP`
paul-soporan Oct 25, 2022
0f719cb
feat: better "variant failed to parse" error message
paul-soporan Oct 25, 2022
deebe14
feat: show the error message when loading rc files
paul-soporan Oct 25, 2022
5e93ead
feat: support per-block indentation levels
paul-soporan Oct 26, 2022
838d3c3
refactor: remove `opt` from `comments` which is already optional
paul-soporan Oct 26, 2022
aee153a
refactor: only show input location
paul-soporan Oct 26, 2022
00adfcb
fix: fix indented block expressions preceded by newlines and comments
paul-soporan Oct 26, 2022
dfe8a87
refactor: extract `block_entry_prefix`
paul-soporan Oct 26, 2022
c9a5427
feat: support single quotes in plain scalars
paul-soporan Oct 26, 2022
12c0d3f
feat: support double quotes in plain scalars
paul-soporan Oct 26, 2022
94c686d
refactor: tweaks
paul-soporan Oct 26, 2022
37b389f
feat: support leading "-" and "?" in plain scalars
paul-soporan Oct 26, 2022
29fa974
feat: support leading ":" in scalars
paul-soporan Oct 26, 2022
668b342
refactor: tweaks
paul-soporan Oct 26, 2022
91ec305
fix: improve colon handling
paul-soporan Oct 26, 2022
95e7edd
fix: support implicit plain scalar keys ending in ":"
paul-soporan Oct 26, 2022
399e4f9
test: tweaks
paul-soporan Oct 26, 2022
c9bdff9
refactor: don't use `serde_json::Value`
paul-soporan Oct 26, 2022
949bcf7
perf: use a `Cow`
paul-soporan Oct 26, 2022
d3454f6
perf: use `Cow` in `IndexMap` too
paul-soporan Oct 26, 2022
21391f4
refactor: use `IndexMap::from`
paul-soporan Oct 26, 2022
ac8eed4
perf: use FNV hasher for `HashMap`s
paul-soporan Oct 27, 2022
4a1d1ae
chore: update serde_json
paul-soporan Oct 27, 2022
e535fb1
chore: re-enable preserve_order
paul-soporan Oct 27, 2022
f4142eb
perf: use Fx hasher
paul-soporan Oct 27, 2022
2795d3f
perf: use cow_replace
paul-soporan Oct 27, 2022
46fa264
perf: use `Cow` in escaped_transform
paul-soporan Oct 27, 2022
586d131
style: use consistent quoting in Cargo.toml
paul-soporan Oct 28, 2022
312def7
perf: use latest wasm-opt
paul-soporan Oct 28, 2022
9e89ac2
Merge branch 'master' into paul/feat/yaml-parser-via-nom
paul-soporan Dec 27, 2022
9e41bfa
chore: revert unneeded change
paul-soporan Dec 27, 2022
58e173c
chore: update rust toolchain
paul-soporan Dec 27, 2022
bbc9a3b
chore: update binaryen
paul-soporan Dec 27, 2022
ce6537d
chore: update deps
paul-soporan Dec 27, 2022
baaed60
Merge remote-tracking branch 'origin/master' into paul/feat/yaml-pars…
paul-soporan Jun 7, 2023
30abaf5
chore: update rust toolchain and binaryen
paul-soporan Jun 7, 2023
2077b93
chore: update wasm-pack
paul-soporan Jun 7, 2023
bbcd942
Merge branch 'master' into paul/feat/yaml-parser-via-nom
paul-soporan Jul 3, 2023
800f084
chore: fix lint
paul-soporan Jul 3, 2023
423a62c
Merge branch 'master' into paul/feat/yaml-parser-via-nom
paul-soporan Jul 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintignore
Expand Up @@ -15,6 +15,7 @@
/packages/*/build/**
/packages/**/*fixtures*/**
/packages/yarnpkg-libzip/artifacts/**
/packages/yarnpkg-parsers/artifacts/**
/packages/plugin-compat/extra/fsevents/fsevents-*.js

# Files generated by Gatsby
Expand Down
13 changes: 10 additions & 3 deletions .github/workflows/integration-workflow.yml
Expand Up @@ -111,9 +111,9 @@ jobs:

- name: 'Check that the grammars are consistent with fresh builds (fix w/ "yarn grammar:all")'
run: |
if [[ $(git diff --name-only "$(git merge-base origin/"$TARGET_BRANCH" HEAD)" HEAD -- 'packages/yarnpkg-parsers/sources/grammars/*.{pegjs,js}' | wc -l) -gt 0 ]]; then
if [[ $(git diff --name-only "$(git merge-base origin/"$TARGET_BRANCH" HEAD)" HEAD -- 'packages/yarnpkg-parsers/{artifacts,sources/grammars}/**/*' | wc -l) -gt 0 ]]; then
node ./scripts/run-yarn.js grammar:all
[[ $(git diff --name-only packages/yarnpkg-parsers/sources/grammars/*.js | tee /dev/stderr | wc -l) -eq 0 ]]
[[ $(git diff --name-only packages/yarnpkg-parsers/sources/grammars/* | tee /dev/stderr | wc -l) -eq 0 ]]
fi
shell: bash
if: |
Expand All @@ -128,13 +128,20 @@ jobs:
if: |
(success() || failure()) && github.event.pull_request != ''

- name: 'Check for linting errors (fix w/ "yarn test:lint --fix")'
- name: 'Check for linting errors (fix w/ "yarn test:lint:fix")'
run: |
node ./scripts/run-yarn.js test:lint
shell: bash
if: |
success() || failure()

- name: 'Check for formatting errors (fix w/ "yarn test:format:fix")'
run: |
node ./scripts/run-yarn.js test:format
shell: bash
if: |
success() || failure()

- name: 'Check for unmet constraints (fix w/ "yarn constraints --fix")'
run: |
node ./scripts/run-yarn.js constraints
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/pr-smart-merge.yml
Expand Up @@ -65,9 +65,9 @@ jobs:
yarn install
fi

if git diff --name-only --diff-filter=U | grep packages/yarnpkg-parsers/sources/grammars/shell.js; then
git checkout --theirs packages/yarnpkg-parsers/sources/grammars/shell.js
yarn grammar:shell
if git diff --name-only --diff-filter=U | grep packages/yarnpkg-parsers/sources/grammars/*.js; then
git checkout --theirs packages/yarnpkg-parsers/sources/grammars/*.js
yarn grammar:all
fi
fi

Expand Down
14 changes: 14 additions & 0 deletions .gitignore
Expand Up @@ -49,3 +49,17 @@ package.tgz
.idea

coverage

# Generated by Cargo; will have compiled files and executables
debug/
target/

# Backup files generated by rustfmt
**/*.rs.bk

# MSVC Windows builds of rustc generate these, which store debugging information
*.pdb

# wasm-pack
wasm-pack.log
pkg/
56 changes: 38 additions & 18 deletions .pnp.cjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 5 additions & 2 deletions .vscode/extensions.json
Expand Up @@ -5,8 +5,11 @@
"recommendations": [
"EditorConfig.EditorConfig",
"dbaeumer.vscode-eslint",
"arcanis.vscode-zipfs"
"arcanis.vscode-zipfs",
"rust-lang.rust-analyzer"
],
// List of extensions recommended by VS Code that should not be recommended for users of this workspace.
"unwantedRecommendations": []
"unwantedRecommendations": [
"rust-lang.rust"
]
}
10 changes: 9 additions & 1 deletion .vscode/settings.json
Expand Up @@ -24,5 +24,13 @@
"search.exclude": {
"**/.pnp.*": true,
"**/.yarn": true
}
},
// Use clippy to lint rust code
"rust-analyzer.checkOnSave.command": "clippy",
// Use rustfmt to format rust code on save
"[rust]": {
"editor.formatOnSave": true
},
// nom uses a lot of function composition which makes the the callable snippets annoying
"rust-analyzer.completion.callable.snippets": "none",
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
36 changes: 36 additions & 0 deletions .yarn/versions/5794200d.yml
@@ -0,0 +1,36 @@
releases:
"@yarnpkg/cli": major
"@yarnpkg/core": major
"@yarnpkg/parsers": major

declined:
- "@yarnpkg/plugin-compat"
- "@yarnpkg/plugin-constraints"
- "@yarnpkg/plugin-dlx"
- "@yarnpkg/plugin-essentials"
- "@yarnpkg/plugin-exec"
- "@yarnpkg/plugin-file"
- "@yarnpkg/plugin-git"
- "@yarnpkg/plugin-github"
- "@yarnpkg/plugin-http"
- "@yarnpkg/plugin-init"
- "@yarnpkg/plugin-interactive-tools"
- "@yarnpkg/plugin-link"
- "@yarnpkg/plugin-nm"
- "@yarnpkg/plugin-npm"
- "@yarnpkg/plugin-npm-cli"
- "@yarnpkg/plugin-pack"
- "@yarnpkg/plugin-patch"
- "@yarnpkg/plugin-pnp"
- "@yarnpkg/plugin-pnpm"
- "@yarnpkg/plugin-stage"
- "@yarnpkg/plugin-typescript"
- "@yarnpkg/plugin-version"
- "@yarnpkg/plugin-workspace-tools"
- "@yarnpkg/builder"
- "@yarnpkg/doctor"
- "@yarnpkg/extensions"
- "@yarnpkg/nm"
- "@yarnpkg/pnpify"
- "@yarnpkg/sdks"
- "@yarnpkg/shell"