Lazy evaluation of Footnotes

[schauder]

Testing Problem

I'm currently doing tests related to date incompabilities with databases.
Basically I'm checking that f(d) == fx(d) where fx is some database operation.
In case of a failure just having the result expected 0 to be 53 is not very helpful.
I'd like to have a table the function results for a range of dates around d.

This can easily done with footnotes, but if database calls are involved this can mean 1000 extra database calls which is sloooooow.

Suggested Solution

Allow footnotes to be lazy by accepting a Supplier<String> instead of a String.

BTW: cool framework.

[jlink]

@schauder Thanks for the suggestion.

In principle, this looks like a useful feature to me. There's one open question, though: When in the lifecycle of a property try should the supplier's get() method be called? There are two possibilities:

• As late as possible: To have the intended effect of not executing unnecessary calls (to a database or another costly resource) the call should happen only if all other lifecycle hooks could throw in their vote if a try is failing or succeeding. That means, however, that extensions for managing resources will already have finished and thereby closed the resource, which then will be accessed and fail.
• As early as possible: This could mean that a possible execution failure (e.g. while committing a transaction) hasn't occurred yet and jqwik will either have to always call get() - making the intended effect of this feature void - or not add the footnote despite a failure that will occur in later lifecycle calls.

I currently don't see an easy way out of this catch 22. Do you?

I wonder if something like

onFailure( 
   () -> assertThat(...),
   () -> Footnotes.add(...)
)

would work. But that looks like it should be part of the assertions library, not of jqwik.

[schauder]

Not sure how others are using Footnotes and extensions, but for my use case it would make most sense when the get gets called as early as possible, i.e. right after a failing the test itself.

[schauder]

Maybe a warning could be logged when lazy footnotes are present and failure only manifests through an extension, that those don't get properly evaluated.

[jlink]

Maybe a warning could be logged when lazy footnotes are present and failure only manifests through an extension, that those don't get properly evaluated.

Maybe that's doable, maybe it would be too complicated.

Would something like

Footnotes.addAfterFailure( () -> "my footnote" )

suffice for your needs?

[schauder]

Absolutely.

[jlink]

Planned for 1.7.2

[jlink]

Released in 1.7.2-SNAPSHOT