interp: fix multi-line shell pattern match equality

[lollipopman]

commit be03afe added the multi-line
flag to the regexp generated for == and =! testing. However, this
flag is confusing as it allows for a regular expression to match one
line out of a multi-line input.

(m) multi-line mode: ^ and $ match begin/end line in addition to
begin/end text (default false)

This means we are now matching against just one line in a multi-line
string, rather than the entire text. Instead, to match Bash's behavior,
we add the s flag to the regexp:

(s) let . match \n (default false)

With this a * in a shell pattern will match \n.

Glob patterns should always match newline characters, so add the s
flag by default when creating a pattern. Alternately we could add the
s flag only when the . meta-character is needed, but that seemed
more fragile. This behavior worked by accident with file globs because
rather than matching with .* we match with [^/]* which does match
\n.

[lollipopman]

@ihar-orca I would love a review, since you submitted the original patch

[ihar-orca]

@lollipopman looks really good!

[mvdan]

Also thinking about the added API - it seems like we do need the added API, because in some cases we want the partial matching to happen. Or perhaps not - if one needs partial matching, perhaps foobar could be fed in as *foobar*, then it will still work as expected.

[lollipopman]

Also thinking about the added API - it seems like we do need the added API, because in some cases we want the partial matching to happen. Or perhaps not - if one needs partial matching, perhaps foobar could be fed in as *foobar*, then it will still work as expected.

I had not thought about inverting the need for whole string matches, by converting each pattern into a whole string match as you suggest. I tried doing some rough benchmarks in Go and whole string matches appear significantly slower on larger texts, as I would expect.

func BenchmarkPartialString(b *testing.B) {
	b.ReportAllocs()
	out, err := exec.Command("man", "bash").Output()
	if err != nil {
		log.Fatal(err)
	}
	manBash := string(out)
	partial := regexp.MustCompile("COPYRIGHT")
	for i := 0; i < b.N; i++ {
		if !partial.MatchString(manBash) {
			b.Fatal(err)
		}
	}
}
func BenchmarkWholeString(b *testing.B) {
	b.ReportAllocs()
	out, err := exec.Command("man", "bash").Output()
	if err != nil {
		log.Fatal(err)
	}
	manBash := string(out)
	wholeString := regexp.MustCompile("^(?s).*COPYRIGHT.*$")
	for i := 0; i < b.N; i++ {
		if !wholeString.MatchString(manBash) {
			b.Fatal(err)
		}
	}
}

[mvdan]

Ah, good point - "whole" matching is slower, so having it opt-in makes sense even if most use cases within shell will want to opt into the option.

[mvdan]

Very nice patch :) Thanks for bearing with me as I review this - it took me a while as I've been travelling.

More than happy to review more patches from you if they are of this quality.

[lollipopman]

Very nice patch :) Thanks for bearing with me as I review this - it took me a while as I've been travelling.

More than happy to review more patches from you if they are of this quality.

Thanks, hopefully I will find the time to keep them coming!