Unique option causes static pattern to fail

[plumdog]

Environment

• OS Version: Arch
• Node.js Version: 12

Actual behavior

When globbing for 3 patterns, if one of the first two has a *, the third is incorrectly unused, but only under Typescript.

Expected behavior

Running the same code under Typescript or plain Node should do the same thing and use all 3 patterns.

Steps to reproduce

$ mkdir fast-glob-three-patterns
$ cd fast-glob-three-patterns
$ npm init --yes

# Create some files to test with
$ touch a.txt b.txt c.txt
# Would expect this code to find all three files we just created
$ CODE='const fg = require("fast-glob"); fg(["a.*", "b.txt", "c.txt"]).then(console.log)'

# Install some things
$ npm install fast-glob@3.2.2 typescript @types/node

$ npx ts-node -e "$CODE"
[ 'b.txt', 'a.txt' ]
# Has failed to find c.txt
# But run the same code but just with node
$ node -e "$CODE"
[ 'b.txt', 'c.txt', 'a.txt' ]
# Has found c.txt

# Now install fast-glob 2.2.7 and try again
$ npm install fast-glob@2.2.7
$ npx ts-node -e "$CODE"
[ 'b.txt', 'c.txt', 'a.txt' ]
$ node -e "$CODE"
[ 'b.txt', 'c.txt', 'a.txt' ]

# Now try fast-glob 3.0.0 and try again
$ npm install fast-glob@3.0.0
$ npx ts-node -e "$CODE"
[ 'b.txt', 'a.txt' ]
$ node -e "$CODE"
[ 'b.txt', 'c.txt', 'a.txt' ]

# Further, running the compilation to create a file, then running the output file directly with node works
$ echo "$CODE" > main.ts
$ npx ts-node main.ts
[ 'b.txt', 'a.txt' ]
$ npx tsc main.ts
$ cat main.js
var fg = require("fast-glob");
fg(["a.*", "b.txt", "c.txt"]).then(console.log);
$ node main.js
[ 'b.txt', 'c.txt', 'a.txt' ]

So I think this is telling me that something was introduced in 3.0.0 that meant that behaviour under typescript is different. Which is really strange and I don't understand why that would be. I sort of assume this is some oddity of my machine/node/typescript/other, so I'm happy to create a repo that demonstrates this a bit more tidily than the above. If someone could look over the above and point out something that I'm doing wrong, or attempt to replicate, that would be wonderful.

[plumdog]

See https://github.com/plumdog/fast-glob-issue-268 for replication using Github Actions.

I thought this was telling me that is was a problem specific to using npx ... as opposed to npm run ... for some reason, but then https://github.com/plumdog/fast-glob-issue-268/runs/688528524 happened, which appears to be a fail using npm run.

Edit: I was unable to replicate the npm run fail, at least using https://github.com/plumdog/fast-glob-issue-268/actions/runs/109151346

[mrmlnc]

Thank you for the detailed description of the issue.

Looks like a bug when using static and dynamic patterns. Related to the unique option.

As workaround you can use unique: false.

[adam-lynch]

My duplicate issue: avajs/ava#2509. I found that it behaved differently when ran in an Ava test.

To reproduce: https://github.com/adam-lynch/ava-fast-glob-issue

[mrmlnc]

I was able to find some free time to study this issue. The problem is that in the filter we add to the index any entry that passes the uniqueness check. We can easily fix the problem by adding to the index only the entries that have passed all filter checks.

fast-glob/src/providers/filters/entry.ts

Lines 17 to 24 in a5595b7

	private _filter(entry: Entry, positiveRe: PatternRe[], negativeRe: PatternRe[]): boolean {
	if (this._settings.unique) {
	if (this._isDuplicateEntry(entry)) {
	return false;
	}
	this._createIndexRecord(entry);
	}

It will probably, fixing this issue may reduce memory consumption on very large samples.

[mrmlnc]

Shipped with 3.2.3.

[plumdog]

My replication repo agrees that 3.2.3 fixes this - https://github.com/plumdog/fast-glob-issue-268/runs/775833545.

@mrmlnc many thanks.