Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: guybedford/es-module-lexer
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 1.4.2
Choose a base ref
...
head repository: guybedford/es-module-lexer
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 1.5.0
Choose a head ref
  • 2 commits
  • 11 files changed
  • 1 contributor

Commits on Mar 25, 2024

  1. Source phase imports (#168)

    guybedford authored Mar 25, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    eaa7e3c View commit details
  2. 1.5.0

    guybedford committed Mar 25, 2024
    Copy the full SHA
    864ca9d View commit details
Showing with 246 additions and 127 deletions.
  1. +18 −3 README.md
  2. +2 −2 chompfile.toml
  3. +5 −5 lib/lexer.asm.js
  4. +3 −3 lib/lexer.emcc.asm.js
  5. BIN lib/lexer.wasm
  6. +1 −1 package.json
  7. +3 −3 src/lexer.asm.js
  8. +120 −104 src/lexer.c
  9. +23 −4 src/lexer.h
  10. +34 −2 src/lexer.ts
  11. +37 −0 test/_unit.cjs
21 changes: 18 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -6,6 +6,8 @@ A JS module syntax lexer used in [es-module-shims](https://github.com/guybedford

Outputs the list of exports and locations of import specifiers, including dynamic import and import meta handling.

Supports new syntax features including import attributes and source phase imports.

A very small single JS file (4KiB gzipped) that includes inlined Web Assembly for very fast source analysis of ECMAScript module syntax only.

For an example of the performance, Angular 1 (720KiB) is fully parsed in 5ms, in comparison to the fastest JS parser, Acorn which takes over 100ms.
@@ -20,6 +22,8 @@ _Comprehensively handles the JS language grammar while remaining small and fast.
npm install es-module-lexer
```

See [types/lexer.d.ts](types/lexer.d.ts) for the type definitions.

For use in CommonJS:

```js
@@ -60,6 +64,10 @@ import { init, parse } from 'es-module-lexer';
// Comments provided to demonstrate edge cases
import /*comment!*/ ( 'asdf', { assert: { type: 'json' }});
import /*comment!*/.meta.asdf;
// Source phase imports:
import source mod from './mod.wasm';
import.source('./mod.wasm);
`;

const [imports, exports] = parse(source, 'optional-sourcename');
@@ -98,10 +106,10 @@ import { init, parse } from 'es-module-lexer';
// Returns -1
exports[2].le;

// Dynamic imports are indicated by imports[2].d > -1
// In this case the "d" index is the start of the dynamic import bracket
// Import type is provided by `t` value
// (1 for static, 2, for dynamic)
// Returns true
imports[2].d > -1;
imports[2].t == 2;

// Returns "asdf" (only for string literal dynamic imports)
imports[2].n
@@ -128,6 +136,13 @@ import { init, parse } from 'es-module-lexer';
// Returns "import /*comment!*/.meta"
source.slice(imports[4].s, imports[4].e);
// ss and se are the same for import meta

// Returns "'./mod.wasm'"
source.slice(imports[5].s, imports[5].e);

// Import type 4 and 5 for static and dynamic source phase
imports[5].t === 4;
imports[6].t === 5;
})();
```

4 changes: 2 additions & 2 deletions chompfile.toml
Original file line number Diff line number Diff line change
@@ -96,7 +96,7 @@ deps = ['src/lexer.h', 'src/lexer.c']
run = """
${{ WASI_PATH }}/bin/clang src/lexer.c --sysroot=${{ WASI_PATH }}/share/wasi-sysroot -o lib/lexer.wasm -nostartfiles \
"-Wl,-z,stack-size=13312,--no-entry,--compress-relocations,--strip-all,\
--export=parse,--export=sa,--export=e,--export=ri,--export=re,--export=is,--export=ie,--export=ss,--export=ip,--export=se,--export=ai,--export=id,--export=es,--export=ee,--export=els,--export=ele,--export=f,--export=ms,--export=__heap_base" \
--export=parse,--export=sa,--export=e,--export=ri,--export=re,--export=is,--export=ie,--export=it,--export=ss,--export=ip,--export=se,--export=ai,--export=id,--export=es,--export=ee,--export=els,--export=ele,--export=f,--export=ms,--export=__heap_base" \
-Wno-logical-op-parentheses -Wno-parentheses \
-Oz
"""
@@ -110,7 +110,7 @@ run = """
${{ EMSDK_PATH }}/emsdk activate 1.40.1-fastcomp
${{ EMSDK_PATH }}/fastcomp/emscripten/emcc ./src/lexer.c -o lib/lexer.emcc.js -s WASM=0 -Oz --closure 1 \
-s EXPORTED_FUNCTIONS="['_parse','_sa','_e','_ri','_re','_is','_ie','_ss','_ip','_se','_ai','_id','_es','_ee','_els','_ele','_f','_ms','_setSource']" \
-s EXPORTED_FUNCTIONS="['_parse','_sa','_e','_ri','_re','_it','_is','_ie','_ss','_ip','_se','_ai','_id','_es','_ee','_els','_ele','_f','_ms','_setSource']" \
-s ERROR_ON_UNDEFINED_SYMBOLS=0 -s SINGLE_FILE=1 -s TOTAL_STACK=4997968 -s --separate-asm -Wno-logical-op-parentheses -Wno-parentheses
# rm lib/lexer.emcc.js
Loading