Proposal: support multiple <meta name="fuzzy">

[nt1m]

The motivation for this is that different browsers might show fuzziness in different ways, some with low maxDifference but high totalPixels (0-50, 0-400), some being the opposite (0-100, 0-50).

When that is the case, multiple fuzzy annotations can be useful, to avoid ending up in the situation where you have to use the maximum in all browsers (0-100, 0-400), which isn't super ideal.

[gsnedders]

c.f. web-platform-tests/wpt#7543 for where we originally added support

web-platform-tests/wpt#7543 (comment) is probably relevant to the original design here:

If some form of fuzziness is introduced, I think it needs to be specified per-engine.

Some engine implementations may know that they expect pixel perfect (or near) results for a test, and wouldn't want to have fuzziness added due to implementation details of a different engine.

In terms of specifying fuzziness, even just per-engine isn't enough. For example, Gecko can specify that fuzziness only occurs on a certain platform, or if a certain preference is enabled.

Instead, it seems like it should be possible to specify a string (or similar) per-engine that is interpreted by the reftest harness for that engine, allowing each engine to customize how it specifies fuzziness.

Except with wpt.fyi we don't have that per-engine expectation data, thus we've increasingly been putting the metadata into the test itself.

[jgraham]

I mean one could have something like maxDifference=0-50;totalPixels=0-400;condition='product == "firefox" and os == "win" and debug' but architecturally getting access to the condition parsing and evaluation code inside wptrunner would be somewhat challenging.

Just allowing any of multiple annotations to apply is easier of course.