Skip to content

Commit

Permalink
feat(html/parser): Improve lexer (#6543)
Browse files Browse the repository at this point in the history
  • Loading branch information
@alexander-akait
alexander-akait committed Dec 1, 2022
1 parent 253f309 commit 49b7e9e
Show file tree
Hide file tree
Showing 12 changed files with 1,185 additions and 714 deletions.
1,302 changes: 661 additions & 641 deletions crates/swc_html_parser/src/lexer/mod.rs
View file

Large diffs are not rendered by default.

22 changes: 22 additions & 0 deletions crates/swc_html_parser/tests/fixture/text/cr/dom.rust-debug
View file
Expand Up @@ -67,3 +67,25 @@
}

"
| "
"
| <div>
| test="test

test

test"
| "
"
| <div>
| test="

test

test

test

"
| "
"
2 changes: 1 addition & 1 deletion crates/swc_html_parser/tests/fixture/text/cr/input.html
View file
@@ -1 +1 @@
<!doctypehtml>Text<div>Text</div><div> Text</div><div> Text Text</div><div>Text Text</div><divtest="test"></div><divdata-q="test"></div><divdata-q='test'></div><divdata-q=test></div><style>.color { color: red; <!-- Test -->}</style>
<!doctypehtml>Text<div>Text</div><div> Text</div><div> Text Text</div><div>Text Text</div><divtest="test"></div><divdata-q="test"></div><divdata-q='test'></div><divdata-q=test></div><style>.color { color: red; <!-- Test -->}</style><div test="testtesttest"></div><div test="testtesttest"></div>
Expand Down
90 changes: 89 additions & 1 deletion crates/swc_html_parser/tests/fixture/text/cr/output.json
View file
Expand Up @@ -2,7 +2,7 @@
"type": "Document",
"span": {
"start": 1,
"end": 305,
"end": 382,
"ctxt": 0
},
"mode": "no-quirks",
Expand Down Expand Up @@ -429,6 +429,94 @@
],
"content": null,
"isSelfClosing": false
},
{
"type": "Text",
"span": {
"start": 305,
"end": 306,
"ctxt": 0
},
"data": "\n",
"raw": "\r"
},
{
"type": "Element",
"span": {
"start": 306,
"end": 341,
"ctxt": 0
},
"tagName": "div",
"namespace": "http://www.w3.org/1999/xhtml",
"attributes": [
{
"type": "Attribute",
"span": {
"start": 311,
"end": 334,
"ctxt": 0
},
"namespace": null,
"prefix": null,
"name": "test",
"rawName": "test",
"value": "test\n\ntest\n\ntest",
"rawValue": "\"test\r\rtest\r\rtest\""
}
],
"children": [],
"content": null,
"isSelfClosing": false
},
{
"type": "Text",
"span": {
"start": 341,
"end": 342,
"ctxt": 0
},
"data": "\n",
"raw": "\r"
},
{
"type": "Element",
"span": {
"start": 342,
"end": 381,
"ctxt": 0
},
"tagName": "div",
"namespace": "http://www.w3.org/1999/xhtml",
"attributes": [
{
"type": "Attribute",
"span": {
"start": 347,
"end": 374,
"ctxt": 0
},
"namespace": null,
"prefix": null,
"name": "test",
"rawName": "test",
"value": "\n\ntest\n\ntest\n\ntest\n\n",
"rawValue": "\"\r\rtest\r\rtest\r\rtest\r\r\""
}
],
"children": [],
"content": null,
"isSelfClosing": false
},
{
"type": "Text",
"span": {
"start": 381,
"end": 382,
"ctxt": 0
},
"data": "\n",
"raw": "\r"
}
],
"content": null,
Expand Down
196 changes: 134 additions & 62 deletions crates/swc_html_parser/tests/fixture/text/cr/span.rust-debug
View file

Large diffs are not rendered by default.

22 changes: 22 additions & 0 deletions crates/swc_html_parser/tests/fixture/text/lf/dom.rust-debug
View file
Expand Up @@ -67,3 +67,25 @@
}

"
| "
"
| <div>
| test="test

test

test"
| "
"
| <div>
| test="

test

test

test

"
| "
"
16 changes: 15 additions & 1 deletion crates/swc_html_parser/tests/fixture/text/lf/input.html
View file
Expand Up @@ -52,4 +52,18 @@
<!-- Test -->
}

</style>
</style>
<div test="test
test
test"></div>
<div test="
test
test
test
"></div>
90 changes: 89 additions & 1 deletion crates/swc_html_parser/tests/fixture/text/lf/output.json
View file
Expand Up @@ -2,7 +2,7 @@
"type": "Document",
"span": {
"start": 1,
"end": 305,
"end": 382,
"ctxt": 0
},
"mode": "no-quirks",
Expand Down Expand Up @@ -429,6 +429,94 @@
],
"content": null,
"isSelfClosing": false
},
{
"type": "Text",
"span": {
"start": 305,
"end": 306,
"ctxt": 0
},
"data": "\n",
"raw": "\n"
},
{
"type": "Element",
"span": {
"start": 306,
"end": 341,
"ctxt": 0
},
"tagName": "div",
"namespace": "http://www.w3.org/1999/xhtml",
"attributes": [
{
"type": "Attribute",
"span": {
"start": 311,
"end": 334,
"ctxt": 0
},
"namespace": null,
"prefix": null,
"name": "test",
"rawName": "test",
"value": "test\n\ntest\n\ntest",
"rawValue": "\"test\n\ntest\n\ntest\""
}
],
"children": [],
"content": null,
"isSelfClosing": false
},
{
"type": "Text",
"span": {
"start": 341,
"end": 342,
"ctxt": 0
},
"data": "\n",
"raw": "\n"
},
{
"type": "Element",
"span": {
"start": 342,
"end": 381,
"ctxt": 0
},
"tagName": "div",
"namespace": "http://www.w3.org/1999/xhtml",
"attributes": [
{
"type": "Attribute",
"span": {
"start": 347,
"end": 374,
"ctxt": 0
},
"namespace": null,
"prefix": null,
"name": "test",
"rawName": "test",
"value": "\n\ntest\n\ntest\n\ntest\n\n",
"rawValue": "\"\n\ntest\n\ntest\n\ntest\n\n\""
}
],
"children": [],
"content": null,
"isSelfClosing": false
},
{
"type": "Text",
"span": {
"start": 381,
"end": 382,
"ctxt": 0
},
"data": "\n",
"raw": "\n"
}
],
"content": null,
Expand Down

1 comment on commit 49b7e9e

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark

Benchmark suite Current: 49b7e9e Previous: 85a03e7 Ratio
es/full/bugs-1 422864 ns/iter (± 11280) 370458 ns/iter (± 27704) 1.14
es/full/minify/libraries/antd 2323894819 ns/iter (± 20013410) 2126071308 ns/iter (± 107396193) 1.09
es/full/minify/libraries/d3 483346882 ns/iter (± 2805590) 444563274 ns/iter (± 14763351) 1.09
es/full/minify/libraries/echarts 1979599222 ns/iter (± 63362725) 1908154550 ns/iter (± 60956281) 1.04
es/full/minify/libraries/jquery 135925412 ns/iter (± 3365155) 138430308 ns/iter (± 4781748) 0.98
es/full/minify/libraries/lodash 152847897 ns/iter (± 2225906) 156331538 ns/iter (± 8091552) 0.98
es/full/minify/libraries/moment 78125952 ns/iter (± 1218335) 79124443 ns/iter (± 4906774) 0.99
es/full/minify/libraries/react 26838248 ns/iter (± 313151) 25589507 ns/iter (± 578651) 1.05
es/full/minify/libraries/terser 370734751 ns/iter (± 11264611) 406248367 ns/iter (± 21821112) 0.91
es/full/minify/libraries/three 681259342 ns/iter (± 29553903) 706062144 ns/iter (± 23583724) 0.96
es/full/minify/libraries/typescript 4323561188 ns/iter (± 498139936) 4221221627 ns/iter (± 121685963) 1.02
es/full/minify/libraries/victory 1020244719 ns/iter (± 22256982) 1024790720 ns/iter (± 19998090) 1.00
es/full/minify/libraries/vue 193502239 ns/iter (± 1648757) 188582772 ns/iter (± 5843675) 1.03
es/full/codegen/es3 40228 ns/iter (± 1253) 34636 ns/iter (± 3755) 1.16
es/full/codegen/es5 40245 ns/iter (± 941) 34774 ns/iter (± 4538) 1.16
es/full/codegen/es2015 40396 ns/iter (± 1005) 35947 ns/iter (± 3677) 1.12
es/full/codegen/es2016 40610 ns/iter (± 1889) 35631 ns/iter (± 3634) 1.14
es/full/codegen/es2017 40382 ns/iter (± 596) 36075 ns/iter (± 6066) 1.12
es/full/codegen/es2018 40306 ns/iter (± 440) 34889 ns/iter (± 2152) 1.16
es/full/codegen/es2019 40558 ns/iter (± 1204) 35648 ns/iter (± 4410) 1.14
es/full/codegen/es2020 40401 ns/iter (± 3637) 35419 ns/iter (± 3829) 1.14
es/full/all/es3 230059006 ns/iter (± 9556870) 209952966 ns/iter (± 11101178) 1.10
es/full/all/es5 217676489 ns/iter (± 6298302) 198033056 ns/iter (± 13179985) 1.10
es/full/all/es2015 175986654 ns/iter (± 4728969) 158388755 ns/iter (± 13187496) 1.11
es/full/all/es2016 175797872 ns/iter (± 5283379) 154823462 ns/iter (± 8485256) 1.14
es/full/all/es2017 174446317 ns/iter (± 5959674) 152894154 ns/iter (± 9587760) 1.14
es/full/all/es2018 171502661 ns/iter (± 4854162) 166113519 ns/iter (± 15015915) 1.03
es/full/all/es2019 170950144 ns/iter (± 5509598) 151687336 ns/iter (± 10630000) 1.13
es/full/all/es2020 164553031 ns/iter (± 6041962) 146928926 ns/iter (± 10998217) 1.12
es/full/parser 865801 ns/iter (± 69938) 757962 ns/iter (± 48486) 1.14
es/full/base/fixer 32351 ns/iter (± 503) 28470 ns/iter (± 3668) 1.14
es/full/base/resolver_and_hygiene 110498 ns/iter (± 2066) 96228 ns/iter (± 11167) 1.15
serialization of ast node 242 ns/iter (± 8) 227 ns/iter (± 37) 1.07
serialization of serde 259 ns/iter (± 15) 225 ns/iter (± 24) 1.15

This comment was automatically generated by workflow using github-action-benchmark.

Please sign in to comment.