Added docs on statistics.

documentation/docs/proptest/statistics.md

@@ -0,0 +1,167 @@
+___---
+id: propteststatistics
+title: Statistics
+sidebar_label: Statistics
+slug: property-test-statistics.html
+---
+
+## Statistics
+
+Soemtimes you may like to know the types of values being generated by Kotest, to ensure that your generators are
+configured in the way you expect. Property test _statistics_ are designed to fulfil this need.
+
+The function `collect` is the entry point to statistcs and is used to count categories of values.
+We use this by invoking it inside a property test with the category we want to increment.
+
+For example, lets say we wanted to gather statistics on the values of `RoundingMode` used by a `BigDecimal`. We would
+invoke `checkAll` as normal, passing the rounding mode to the collect function.
+
+```kotlin
+checkAll(Arb.enum<RoundingMode>(), Arb.bigDecimal()) { mode, decimal ->
+   collect(mode)
+   // test here
+}
+```
+
+Now after the test is completed, Kotest will output the test name, and the counts/percentages for each category:
+
+```
+Statistics: [collecting stats] (1000 iterations, 1 args)
+
+HALF_DOWN                                                     142 (14%)
+HALF_UP                                                       141 (14%)
+CEILING                                                       132 (13%)
+FLOOR                                                         122 (12%)
+UP                                                            119 (12%)
+UNNECESSARY                                                   119 (12%)
+HALF_EVEN                                                     118 (12%)
+DOWN                                                          107 (11%)
+```
+
+The category we use does not have to be an enum. It can be any object, and you can wrap in conditionals if you want more
+control. For example:
+
+```kotlin
+checkAll(Arb.int()) { k ->
+   when {
+     k % 2 == 0 -> collect("EVEN")
+     else -> collect("ODD")
+   }
+   // test here
+}
+```
+
+## Labels
+
+Sometimes you may wish to have orthogonal sets of statistics. For example,
+for a simple number test, we might want to confirm that a certain percentage are even numbers, and a certain percentage
+are negative. One way would be to have EVEN_POS, EVEN_NEG, ODD_POS, ODD_NEG:
+
+```kotlin
+checkAll(Arb.int()) { k ->
+   when {
+     k > 0 && k % 2 == 0 -> collect("EVEN_POS")
+     k % 2 == 0 -> collect ("EVEN_NEG")
+     k > 0 -> collect("ODD_POS")
+     else -> collect("ODD_NEG")
+   }
+   // test here
+}
+```
+
+This gives us one set of outputs:
+
+```
+EVEN_POS                                                       142 (27%)
+EVEN_NEG                                                       141 (23%)
+ODD_POS                                                        132 (24%)
+ODD_NEG                                                        122 (26%)
+```
+
+However, as the combinations grow this will become unwieldy, so Kotest supports labelled statistics. You can think of
+this as distinct sets of statistics. To use labels, just pass the label name as the first arg to the collect method.
+
+```kotlin
+checkAll(Arb.int()) { k ->
+  when {
+    k % 2 == 0 -> collect("even_odd", "EVEN")
+    else -> collect("even_odd", "ODD")
+  }
+  when {
+    k > 0 -> collect("pos_neg", "POS")
+    else -> collect("pos_neg", "NEG")
+  }
+  // test here
+}
+```
+
+
+Now, Kotest will output multiple sets of statistics, with the label name in the title:
+
+```
+Statistics: [collecting labelled stats] (1000 iterations, 1 args) [even_odd]
+
+ODD                                                           520 (52%)
+EVEN                                                          480 (48%)
+
+
+Statistics: [collecting labelled stats] (1000 iterations, 1 args) [pos_neg]
+
+NEG                                                           527 (53%)
+POS                                                           473 (47%)
+```
+
+
+### Report Mode
+
+By default, statistics are printed for every property test. There are four modes which can be configured using the global configuration object `PropertyTesting`.
+
+The possible options are:
+
+| Mode                                                                | Function                                   |
+|---------------------------------------------------------------------|--------------------------------------------|
+| PropertyTesting.statisticsReportMode = StatisticsReportMode.OFF     | disable all statistics reporting           |
+| PropertyTesting.statisticsReportMode = StatisticsReportMode.OFF     | enables all statistics reporting           |
+| PropertyTesting.statisticsReportMode = StatisticsReportMode.SUCCESS | output statistics only on successful tests |
+| PropertyTesting.statisticsReportMode = StatisticsReportMode.FAILED  | output statistics only on failed tests     |
+
+
+### Checking Coverage of Statistics
+
+If you wish to programmatically assert that certain values are being generated, then you can use the specify constraints
+that must be met.
+
+For example, in our previous rounding example, we can check that at least 10% of inputs are covering HALF_DOWN, and 10%
+are covering FLOOR using `withCoveragePercentages`:
+
+```kotlin
+withCoveragePercentages(mapOf(RoundingMode.HALF_DOWN to 10.0, RoundingMode.FLOOR to 10.0)) {
+  checkAll(Arb.enum<RoundingMode>(), Arb.bigDecimal()) { mode, decimal ->
+    collect(mode)
+    // use the mode / decimal
+  }
+}
+```
+
+If we want to check by absolute numbers rather than percentages, we can use `withCoverageCounts`:
+
+```kotlin
+withCoverageCounts(mapOf(RoundingMode.HALF_DOWN to 75, RoundingMode.FLOOR to 75)) {
+  checkAll(Arb.enum<RoundingMode>(), Arb.bigDecimal()) { mode, decimal ->
+    collect(mode)
+    // use the mode / decimal
+  }
+}
+```
+
+### Custom Reports
+
+You can customize the report format, or generate reports from the raw data, by using your own instance of `StatisticsReporter`.
+This is configured via the global configuration object `PropertyTesting`.
+
+For example:
+
+```
+object MyStatisticsReporter : object : StatisticsReporter { ... }
+PropertyTesting.statisticsReporter = MyStatisticsReporter
+```

documentation/sidebars.js

@@ -15,6 +15,7 @@ module.exports = {
       "proptest/proptestconfig",
       "proptest/customgens",
       "proptest/shrinking",
+      "proptest/statistics",
       "proptest/globalconfig",
       "proptest/arrow",
       "proptest/date_gens",