Custom Examples and fc.context() are incompatible 🐛

[tskj]

🐛 Bug Report

Like this issue describes, it is currently not possible to use fc.context() (in my usecase for logging) with custom examples. Both custom examples and logging are extremely useful and good features, and as a user I would expect them to either be compatible, or for the docs to clearly warn about their incompatibility.

To Reproduce

Like the code in the aforementioned issue describes, attempt to use custom examples in a property that uses fc.context():

fc.assert(
  fc.property(fc.string(), fc.context(), (str, ctx) => /* ... */),
  { examples: [["x", /* what here? */]] }
);

Expected behavior

Ideally it would always be possible to copy-paste the counter examples that fast-check outputs when finding a failing test case into your test code.

That is the behavior that new users expect, and would be a very good user experience. Not being able to do that feels broken and seems like a bug in the library.

Suggestions

There are many ways to square these incompatibilities, one very crude way to fix this would be to use the suggested code from the other issue:

const buildContext = () => fc.sample(fc.context(), {numRuns:1})[0];

and simply output this along with the counter examples when fast-check sees that one of the example values is a Context. Kind of like the following:

...
Property failed after 1 test:
{ seed: ...
Counterexample: [ '', () => fc.sample(fc.context(), {numRuns:1})[0] ]
Shrunk 0 time(s)
...

I'm sure there also are better technical solutions, but which presumably require more work. Either way, I think a prinicple to strive for is for the counter examples that are outputted by fast-check itself to not be broken.

If we don't want to fix the library itself, I think at least there should be added to the docs an explanation of how the user can fix it themselves. I have created a PR for this here, using the buildContext function.

As an alternative if we don't want the buildContext implementation to be an "official" suggestion, I have also created a different PR here, which at the very least explains the incompatibility and warns the user.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[dubzzz]

Let's close it for now. The documentation has been adapted for future users