Is there anyway to pass the threshold to the Golden Test comparator?

[jamontes79]

Description
I want to use a custom Golden Test comparator to use a threshold for comparing the generated images. Is that possible? Is there any documentation?

I've been searching in the closed issues and I've found some of them closed but nothing explaining how to use it.

Thanks in advance

[tomarra]

Notes

• Looks like this is supported already starting in 0.17.0 Golden file comparator can't be extended to define a custom behavior  #741
• Need to update the documentation to state this

[jamontes79]

Any news about this? Thanks

[alestiago]

Hi @jamontes79 , sorry for the late reply, I'll assign this to myself and will hopefully be able to provide documentation and clarity to the question the upcoming week.

[jamontes79]

Hi @jamontes79 , sorry for the late reply, I'll assign this to myself and will hopefully be able to provide documentation and clarity to the question the upcoming week.

No worries. Thanks a lot!

[alestiago]

@jamontes79 you should be able to use the goldenFileComparator property in the flutter_test library to achieve so. There is no additional code that you'll need to provide to make it work with very_good test, since the optimized test already considers and respects explicitly the goldenFileComparator.

Here's an example on how a test with some tolerance would look like:

import 'dart:typed_data';

import 'package:flutter/widgets.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  testWidgets('renders as expected', (WidgetTester tester) async {
    final previousGoldenFileComparator = goldenFileComparator;
    goldenFileComparator = _TolerantGoldenFileComparator(
      // Replace with your test file path:
      Uri.parse('test/widget_test.dart'),
      precisionTolerance: 0.01,
    );
    addTearDown(() => goldenFileComparator = previousGoldenFileComparator);

    await tester.pumpWidget(const ColoredBox(color: Color(0xff00ff00)));

    await expectLater(
      find.byType(ColoredBox),
      matchesGoldenFile('my_golden.png'),
    );
  });
}

class _TolerantGoldenFileComparator extends LocalFileComparator {
  _TolerantGoldenFileComparator(
    super.testFile, {
    required double precisionTolerance,
  }) : _precisionTolerance = precisionTolerance;

  /// How much the golden image can differ from the test image.
  ///
  /// It is expected to be between 0 and 1. Where 0 is no difference (the same image)
  /// and 1 is the maximum difference (completely different images).
  final double _precisionTolerance;

  @override
  Future<bool> compare(Uint8List imageBytes, Uri golden) async {
    final result = await GoldenFileComparator.compareLists(
      imageBytes,
      await getGoldenBytes(golden),
    );

    final passed = result.passed || result.diffPercent <= _precisionTolerance;
    if (passed) {
      result.dispose();
      return true;
    }

    final error = await generateFailureOutput(result, golden, basedir);
    result.dispose();
    throw FlutterError(error);
  }
}

Note that it is good practice to reset the value back to its original to avoid your comparator leaking into other golden tests when running parallelised tests. If you want to define a global configuration read into using the flutter_test_config.dart.

---

I've seen there's a lack of documentation online regarding this practice, hence I'll try to follow-up this item with documentation on our tooling (Very Good CLI), Flutter documentation, and Stack Overflow answers.