Fix a bug where incremental parsing could inappropriately reuse a node when it ended in a repeat or optional part, and was followed by a sequence of skipped nodes longer than 25 characters.
The new InputStream.acceptTokenTo method allows a tokenizer to create a token that ends at some absolute position.
Further reduce maximum parse depth—the old limit could still cause the tree building function to overflow the stack on V8 when all rules on the stack produce actual visible tree nodes.
The parser will now emit an error node when it reaches the end of the top rule but not the end of the input, and restarts parsing the top rule.
Fix an issue that could, when error-recovery happened after a skipped token, cause the parser to emit nodes whose start position was after their end position.
Reduce tree depth limit to avoid V8 overflowing its stack.
When parsing input ranges with gaps, don't treat the token after a gap as covering the entire gap.
Make this package usable in TypeScript setups using node16/nodenext resolution.
Fix another issue in the way local token groups and input gaps interact.
Fix an issue that caused incorrect trees to be emitted when a forced reduction happened inside a skipped term.
Fix a bug where the parser could skip characters while tokenizing in a local token group.
Fix an issue where context-tracking parsers, in specific circumstances, could create incorrectly nested trees.
Fix an issue where some grammars could make the parser unable to force reductions on invalid input, leading to trees that were missing some wrapping nodes around the broken input.
Fix tokenizing of @else tokens that span separate input ranges.
Fix a crash when parsing a non-atomic skippable at the top of a nested parse.
Fix a bug that caused the tree-depth limit on left-associative rules to be applied to rules generated for repeat operators.
Place a limit on the amount of (non-recovering) parse stacks active at the same time to avoid exponential slowdown on inputs that nest ambiguous constructs.
Reduce the maximum left-associative nesting depth, since the old value could still cause Chrome (but not Firefox) to overflow its call stack.
Fix a bug that could corrupt the parse stack when parsing a non-single-token skippable expression from the initial parse state.
Avoid the problem of huge numbers of repeated left-associative operator applications creating a stack too deep to recurse on by detecting such parses and force-reducing out of them.
Add support for grammars that use @local tokens declarations.
Fix a bug that made Stack.canShift incorrectly return true in some circumstances.
Fix a tokenizer bug that broke tokenizing of the character \uffff.
Fix a regression in the way tokenizers read input when parsing multiple separate ranges.
Fix an issue that could cause the tokenizer to crash when tokens ended directly on a gap in the input ranges.
Include LRParser.deserialize in the TypeScript declarations.
Fix a bug that broke ParserConfig.specializers (requires @lezer/generator 1.1.1).
External specializers can now be replaced when reconfiguring an LRParser.
Support the output produced by the new @eof marker in lezer-generator.
First stable version.
Don't crash when a non-Node environment defines a process global that doesn't have an env property.
Fix a bug that could cause partially-parsed nodes to be reused when they ended in skipped nodes, when parsing incrementally.
Fix an issue that could make the parser merge adjacent error nodes even when they had different parent nodes.
Fix an issue that caused error nodes created for inserted tokens to sometimes end up in the wrong parent node.
Avoid the need for generated parser files to directly import @lezer/common.
Fix a bug that caused reductions that didn't consume anything to sometimes end up outside their parent node in the tree.
Fix a bug that could cause some kinds of composite skipped expressions to not be parsed correctly.
Make sure tree depth does not grow unbounded, so that recursive tree traversal is safe from overflowing the stack.
Be less agressive about pruning long-running GLR parse splits.
Fix a bug that caused node lookahead to be one less than it should be.
Fix an issue that could cause the parser to, when recovering, parse beyond the position given to stopAt.
Fix a mistake in the way forced reductions are checked.
Fix crashes or infinite recursion caused by applying forced reductions in situations where they were not valid.
Fix a bug where the parse position could get corrupted during a parse, leading to crashes or nonsensical output.
Fix an inconsistency in the .d.ts emitted by the build.
LRParser.configure now takes a contextTracker option to replace the context tracker used by the parser.
The module's name changed from lezer to @lezer/lr.
The Parser class was renamed to LRParser (Parser is now the abstract class that all parsers extend).
Nested parsing is no longer handled inside the LR parser (instead, it is done by using parseMixed from the @lezer/common package as a wrapper).
External tokenizers are passed different arguments and get a different input stream abstraction (to make accidental uncontrolled lookahead and lookbehind, which were easy ways to break incremental parsing, harder).
Drops support for Stack.startOf (which is no longer useful without lookbehind).
Context trackers also get passed different arguments, following the changed input stream format.
This package no longer re-exports bindings from @lezer/common.
Slightly prefer deleting token to inserting them to avoid far-fetched parses.
Fix a problem where node reuse didn't take the amount of look-ahead done by the tokenizer into account, and could reuse nodes whose content would tokenize differently due to changes after them.
Track tokenizer lookahead to fix a problem where some incremental parses would produce incorrect results.
LR parsers now support a wrapper configuration option that allows you to inject additional logic by wrapping the PartialParse object it returns.
Fix a bug with overeager reuse of nodes on change boundaries.
Parser instances now have a topNode property that holds the type of the parser's top node.
Context trackers can now disable strictness in node reuse.
Add support for context trackers.
Fix versions of lezer packages depended on.
Parser.group is now called Parser.nodeSet.
Nested parsers now work differently. They don't have to be Lezer parsers, but have to produce object conforoming to the PartialParse interface. The interface with which non-trivial nested parsers are specified also changed—see the NestedParser type.
Parser objects no longer have a topType property (scan their node set for types with isTop set instead).
Parser objects no longer have withProps, withNested, and withTokenizer methods (use configure instead).
Both Parser.parse and Parser.startParse now take an optional start position as second parameter and an optional parse context as third. startParse returns an instance of the PartialParse interface instead of the old ParseContext class (whose name is now used for something else). Parse options are no longer passed to these methods, but configured in advance through Parser.configure.
During incremental parsing, instead of passing a tree as the cache option, reusable subtrees (see TreeFragment from lezer-tree) are now retrieved from the fragments property of the parse context object, if provided.
Parser.parse and Parser.startParse no longer take an options parameter. Instead, bufferLength and strict can be configured with Parser.configure now, and the start position and context are passed as optional arguments.
The InputStream type has been replaced by Input from the lezer-tree package (which has the same interface but a more appropriate name).
The Parser class now has a configure method that is used to create a parser instance with a more specific configuration.
Fix an infinite loop in incremental parsing when repeatedly reusing a zero-length cached node.
Follow the change from Subtree to TreeCursor in lezer-tree.
The serialized parser format changed.
Stack.startOf now returns null, rather than -1, when it can't find the given element.
Stack.startOf now takes an optional second argument that allows you to select a match beyond the innermost one.
Fix lezer depencency versions
Fix an infinite loop that was sometimes hit during error recovery.
Follow a breaking change in the way repeat nodes are represented.
Support the new action table format that allows sharing between states.
Parser objects now have a withTokenizer method that can be used to replace external tokenizers.
Fix a bug that caused the value returned by ParseContext.badness to be much higher than intended.
Stack.ruleStart will now ignore repeat rules and node-less rules when determining the inner rule.
Work around a failure mode where error-recovery got stuck in an end-of-grammar state and thus could not continue meaningfully parsing anything by restarting such states back to their initial state.
External tokenizers can now provide an extend flag to allow their tokens to be used alongside tokens produced by other tokenizers.
Add support for dynamic precedences.
Fixes an issue where repeated error recovery could lead to a tree so deep that recursive functions on it would overflow the stack.
Add support for grammar dialects.
Add support for external specializers.
Stacks now have a parser accessor that gets you the active parser instance.
No longer list internal properties in the type definitions.
Follow changes in the serialized parser format.
The way different tokenizers are combined is now slightly different. The first one to return a token wins, even if that token has no actions in the current state. The old behavior, where further tokenizers are tried until actions are found, can be enabled for a given tokenizer by setting its fallback flag.
Fix accidental use of non-ES5 library methods.
Upgrade to 0.9 parser serialization
Publish less useless cruft to npm, reducing package size.
Fix a bug in Stack.startOf that made it fail to find rules that were actually on the stack in many situations.
Make the package load as an ES module on node
The package now provides an ES6 module.
You can now find the top node type parsed by the parser through its topType property.
The serialized parser format changed.
Add support for multiple @top rules through the top parse option.
Tweak recovery cost for forced reductions to prefer those to other recovery strategies.
More agressively reuse cached nodes.
This now consumes the adjusted parser output of lezer-generator 0.7.0.
Rewrite the way the parser advances and recovers from errors, to more rigorously address a number of infinite loops and poor recovery behaviors.
Parse context objects now have a badness property that you can use to estimate just how poorly the input matches the grammar.
Fix an issue where the parser will sometimes continue, and even pick as result, a parse with error recovery even though there are error-free parses available.
Fix a mistake in our binary heap implementation that would cause stacks to be ordered incorrectly.
Fix an issue where the Stack.startOf method would ignore the top frame of the stack.
Fix an issue where the parser would loop infinitely when leaving a nested parse in some circumstances.
Fix an infinite loop on incomplete input at end of file that occurred for some types of mutually recursive rules.
Parser instances now have a hasNested property that tells you whether they nest grammars.
Fix an infinite loop where error recovery keeps finding zero-length tokens and imagining it's making progress.
Don't rely on additional data stored in the parse table during recovery (shrinking the parse tables).
Fix a crash that could occur when starting a nested parse when there were multiple active stacks.
Fix an issue where error nodes would sometimes not be merged.
Don't reuse cached tokens for states that have a different token group.
The on-disk parse table format changed again.
Don't treat reused nodes as if they are error terms when merging errors.
Add badness penalty for forced reductions at end of input.
Fix several infinite loops around forced reductions.
Don't move error nodes out of reduces.
Add a Parser.withProps method for extending a parser with new node props.
Emits lezer-tree 0.3.0 style trees with NodeType type objects.
Parser.deserialize's interface changed (now taking an object rather than a long list of parameters).
Don't include lezer-tree inline in dist/index.js.
The output tree now uses tags, rather than term names, to identify its nodes.
Export Tag data structure from lezer-tree.
Support per-grammar global tag suffixes in Parser.deserialize.
Grammars no longer have ids.
Removes export of allocateGrammarID and TagMap.
Actually include the .d.ts file in the published package.
First documented release.