New tool: oxc_module_lexer (#2650)

# Oxc Module Lexer

This is not a lexer. The name "lexer" is used for easier recognition.

## [es-module-lexer](https://github.com/guybedford/es-module-lexer)

Outputs the list of exports and locations of import specifiers,
including dynamic import and import meta handling.

Does not have any
[limitations](https://github.com/guybedford/es-module-lexer?tab=readme-ov-file#limitations)
mentioned in `es-module-lexer`.

I'll also work on the following cases to make this feature complete.

- [ ] get imported variables
guybedford/es-module-lexer#163
- [ ] track star exports as imports as well
guybedford/es-module-lexer#76
- [ ] TypeScript specific syntax
- [ ] TypeScript `type` import / export keyword

## [cjs-module-lexer](https://github.com/nodejs/cjs-module-lexer)

- [ ] TODO

## Benchmark

This is 2 times slower than `es-module-lexer`, but will be significantly
faster when TypeScript is processed.

The difference is around 10ms vs 20ms on a large file (700k).

Cargo.toml

@@ -84,6 +84,9 @@ oxc_prettier       = { path = "crates/oxc_prettier" }
 oxc_tasks_common    = { path = "tasks/common" }
 oxc_language_server = { path = "crates/oxc_language_server" }
 
+napi        = { version = "2" }
+napi-derive = { version = "2" }
+
 assert-unchecked          = { version = "0.1.2" }
 bpaf                      = { version = "0.9.9" }
 bitflags                  = { version = "2.4.2" }

crates/oxc_module_lexer/Cargo.toml

@@ -0,0 +1,28 @@
+[package]
+name                   = "oxc_module_lexer"
+version                = "0.0.0"
+publish                = false
+authors.workspace      = true
+description.workspace  = true
+edition.workspace      = true
+homepage.workspace     = true
+keywords.workspace     = true
+license.workspace      = true
+repository.workspace   = true
+rust-version.workspace = true
+categories.workspace   = true
+
+[lints]
+workspace = true
+
+[lib]
+test    = false
+doctest = false
+
+[dependencies]
+oxc_ast  = { workspace = true }
+oxc_span = { workspace = true }
+
+[dev-dependencies]
+oxc_allocator = { workspace = true }
+oxc_parser    = { workspace = true }

crates/oxc_module_lexer/README.md

@@ -0,0 +1,24 @@
+# Oxc Module Lexer
+
+This is not a lexer. The name "lexer" is used for easier recognition.
+
+## [es-module-lexer](https://github.com/guybedford/es-module-lexer)
+
+Outputs the list of exports and locations of import specifiers, including dynamic import and import meta handling.
+
+Does not have any [limitations](https://github.com/guybedford/es-module-lexer?tab=readme-ov-file#limitations) mentioned in `es-module-lexer`.
+
+- [ ] get imported variables https://github.com/guybedford/es-module-lexer/issues/163
+- [ ] track star exports as imports as well https://github.com/guybedford/es-module-lexer/issues/76
+- [ ] TypeScript specific syntax
+- [ ] TypeScript `type` import / export keyword
+
+## [cjs-module-lexer](https://github.com/nodejs/cjs-module-lexer)
+
+- [ ] TODO
+
+## Benchmark
+
+This is 2 times slower than `es-module-lexer`, but will be significantly faster when TypeScript is processed.
+
+The difference is around 10ms vs 20ms on a large file (700k).

crates/oxc_module_lexer/examples/module_lexer.rs

@@ -0,0 +1,46 @@
+use std::{env, path::Path};
+
+use oxc_allocator::Allocator;
+use oxc_module_lexer::ModuleLexer;
+use oxc_parser::Parser;
+use oxc_span::SourceType;
+
+// Instruction:
+// * create a `test.js`
+// * `just example module_lexer
+
+fn main() -> Result<(), String> {
+    let name = env::args().nth(1).unwrap_or_else(|| "test.js".to_string());
+    let path = Path::new(&name);
+    let source_text = std::fs::read_to_string(path).map_err(|_| format!("Missing '{name}'"))?;
+    let allocator = Allocator::default();
+    let source_type = SourceType::from_path(path).unwrap();
+    let ret = Parser::new(&allocator, &source_text, source_type).parse();
+
+    println!("source:");
+    println!("{source_text}");
+
+    for error in ret.errors {
+        let error = error.with_source_code(source_text.clone());
+        println!("{error:?}");
+        println!("Parsed with Errors.");
+    }
+
+    let ModuleLexer { imports, exports, facade, has_module_syntax } =
+        ModuleLexer::new().build(&ret.program);
+
+    println!("\nimports:");
+    for import in imports {
+        println!("{import:?}");
+    }
+
+    println!("\nexports:");
+    for export in exports {
+        println!("{export:?}");
+    }
+
+    println!("\nfacade: {facade}");
+    println!("has_module_syntax {has_module_syntax}");
+
+    Ok(())
+}