String#matchAll should return iterable of RegExpExecArray instead of RegExpMatchArray

[ZhangYiJiang]

TypeScript Version: 3.7 (issue is with lib typing)

Search Terms: regex, matchall, es2020

Code

const matches = Array.from('xxx'.matchAll(/x/g));
matches[0].index; // TS reports index may be undefined, when it should always be defined

Currently matchAll returns an iterable of RegExpMatchArray. However, the index and input properties on RegExpMatchArray are optional because String#match returns a match object without those if the regex has the global flag (#35157).

matchAll on the other hand doesn't have the same behavior, so it should use RegExpExecArray where both of those properties are non-optional (or maybe create a RegExpMatchAllArray type)

> Array.from('xxx'.matchAll(/x/g)) 
[
  [ 'x', index: 0, input: 'xxx', groups: undefined ],
  [ 'x', index: 1, input: 'xxx', groups: undefined ],
  [ 'x', index: 2, input: 'xxx', groups: undefined ]
]
> Array.from('xxx'.matchAll(/a/g)) 
[]
> Array.from('aaa'.matchAll(/x/g)) 
[]

Expected behavior:

input and index on the match objects are not optional

Actual behavior:

TS reports that input and index may be optional

Playground Link: (Can't use playground since lib cannot be configured)

Related Issues: #30936 introduced String#matchAll

[RyanCavanaugh]

We don't have any mechanism for carrying around the flags of a RegExp, so the current definitions err on the side of making you assert that those properties will be there in cases where you know that the regexp is in fact global.

[ZhangYiJiang]

I believe the global flag changing the return value only applies to String#match. All objects returned from String#matchAll will have the two properties, without any exceptions

[sholladay]

TypeScript doesn't need to inspect or keep track of flags in this case. A regex given to .matchAll() will always be global. JavaScript throws an error if you attempt to pass it a non-global regex. Thus, unlike the return value of .match(), the return value of .matchAll() is always guaranteed to have .index and .input.

I think the team's decision for how to type .match() makes sense given that TypeScript has a limited understanding of regex flags and .match() has a return type that is conditional on those flags. However, that's a completely unrelated problem and discussion to this issue. Regex flags and limitations surrounding them are irrelevant to .matchAll(), as the only possible return value from .matchAll() is a type where .index and .input are required (always present and non-nullable).

[clemyan]

According to the spec, String#matchAll actually executes with the semantics of RegExp#exec instead of String#match.

So, OP is correct, the return type of String#matchAll should be IterableIterator<RegExpExecArray>, not the current IterableIterator<RegExpMatchArray>.

[kputnins]

Any updates on this?

[younesaassila]

This is still an issue in 2023

[RyanCavanaugh]

We still don't have a way to fix it!

[younesaassila]

@RyanCavanaugh Sure, but why have you closed the issue? Just because "the mechanism isn't there" doesn't mean the problem is fixed

[RyanCavanaugh]

There's an infinite number of things that we don't have the mechanics to solve. Issues are open if they are actionable; this is the definition we use

[clemyan]

We still don't have a way to fix it!

Untrue.

TypeScript/src/lib/es2020.string.d.ts

Line 9 in 1481750

matchAll(regexp: RegExp): IterableIterator<RegExpMatchArray>;

Simply changing this line to

    matchAll(regexp: RegExp): IterableIterator<RegExpExecArray>;

will fix this.

---

According to the spec, String#matchAll actually executes with the semantics of RegExp#exec instead of String#match.

So, OP is correct, the return type of String#matchAll should be IterableIterator<RegExpExecArray>, not the current IterableIterator<RegExpMatchArray>.

[NilsIrl]

Can this be re-opened? This was closed by mistake. This is actionable and was never blocked on there being a way to fix it (because there always has been).

[DanielRosenwasser]

It seems like there was a misreading here - there has been a longstanding issue around typing match, but matchAll - while not being able to reject non-global regexps - will only function when given one, and can have its return type amended.

At least that's what we've gathered revisiting this.

[sholladay]

@DanielRosenwasser yes, exactly!

While it would certainly be a welcome improvement for TypeScript to reject non-global regexps being passed to .matchAll(), that's a separate, unrelated issue.

This issue is strictly about the return type of .matchAll(), which does not depend on the function's arguments whatsoever. Thus, TypeScript does not need to know or care about regexp flags to fix this.

TypeScript can safely assume that .matchAll() always returns a type whose elements have .index and .input. Right now, it lies and says they may not exist.