Use property-based testing in the compiler source to ensure correctness

[DanikVitek]

What's good about property-based testing is that it somewhat enforces writing tests to be implementation-independent and allows testing many random scenarios against a set of "properties", hence the name.

https://lib.rs/crates/proptest is a good library for that. If a test fails, it tries to find the smallest failing case to help diagnose and then stores the failed cases, starting the next test run with their reevaluation.

[DanikVitek]

Automate Your Way to Better Code: Advanced Property Testing (with Oskar Wickström)

[lpil]

Hello! Could you share specific examples of where we could benefit from property testing over our existing snapshot tests and how we would implement them? Thank you

[DanikVitek]

Well, I was reading through the PR #2731 and I noticed that the test there was a perfect subject for the property-based testing,
One could calculate the range, based on the function name and module path, based on the module's name.
That's just one example, and not a merged one, I'll look for more in the actual code.

[lpil]

I think to justify the addition we'd need to find utility beyond a few helper functions. Having a large amount of use similar to how we use snapshot and example tests would be fab.

[DanikVitek]

Here is an example commit for property-based testing of parsing of names and integers
DanikVitek@b1f5e30

[lpil]

What would this enable us to do faster or more easily? If we're to add more complexity into our development workflow we want to get some measurable benefit from it.

I believe there is huge potential in prop testing for us, but I don't have an application for it yet that gives us benefits. That is what we need to find.

[soraros]

Erlang has a great property based testing lib by John Hughes, the originally author of Haskell's QuickCheck. You can just write a gleam wrapper for that, I suppose.

[DanikVitek]

Nice! Though I was talking more about the testing of gleam compiler itself. I haven't clarified this.