WIP: add reporters

[43081j]

@lukeed this is the kind of thing i meant in #52

this is just an example/WIP but hopefully explains the gap we're trying to fill. arguably i doubt many of us care about losing a few bytes in order to get this functionality. the speed is what we care about and that pretty much exists solely because of the simplicity of the current implementation, so i do want to avoid blowing that up.

basically, the suggestion here is that we introduced (implicitly):

interface Reporter {
  start(): void;
  suite(ctx: Context, name: string): void;
  pass(ctx: Context, name: string): void;
  fail(ctx: Context, name: string, err: Error): void;
  suiteEnd(ctx: Context, name: string, passed: number, failed: number, errors: string): void;
  file(name: string, passed: number, total: number): void;
  end(total: number, passed: number, skipped: number, duration: number): void;
}

some thought would need putting into how we pass a reporter in. right now i just hacked in a default one which probably doesn't work right

[codecov-io]

Codecov Report

Merging #107 (23370a8) into master (1d75357) will decrease coverage by 0.90%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##           master     #107      +/-   ##
==========================================
- Coverage   89.65%   88.75%   -0.91%     
==========================================
  Files           4        4              
  Lines         319      329      +10     
==========================================
+ Hits          286      292       +6     
- Misses         33       37       +4     

Impacted Files	Coverage Δ
src/index.js	73.39% <75.00%> (-1.36%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1d75357...23370a8. Read the comment docs.

[lukeed]

Hi - thank you, but this is not at all on the agenda.
I'll have a closer look tomorrow once I'm actually on a computer, but I feel compelled to tell you my thoughts before you spend more time on this.

The JSON issue is left open because that should be all the flexibility one needs to gather results from an outside PoV. In the time that it's been open, there's been very little interest (in reporters at all, actually).

There may be a case made for TAP, but even then, every one of the few conversations I've had about output configurability has been rooted in vanity -- and that's not a valid reason for adding code here IMO

[43081j]

Don't worry about time spent. What doesn't end up in uvu will just end up in a library which consumes the output of uvu most likely.

uvu is still relatively small and hasn't gained much traction yet, so i don't think you can yet claim there's been no interest in any given issue.

currently you output an arbitrary format you made up. in order to be useful in real world projects, it needs an output that can be consumed by other existing tools (for CI). This means one of the following:

• output JSON so we can build tooling around uvu which consumes that and outputs the correct format
• expose a reporter interface so uvu can report via something which outputs the correct format

one way or another, the current design is inadequate IMO for real-world usage as it can't be tied into CI workflows.

personally i'd go with option 2 because it is by far the cleanest solution. so when you get chance, could you explain why you wouldn't? again, this package is fast because it is simple and thats what makes it attractive. you can maintain that while still exposing reporters as a concept.

would be very interested to know the reasoning for avoiding it.

in actual fact, too, i've contributed migrations to uvu to several repos/orgs lately and most only consume your assertion library because uvu isn't of any use in CI reporting (we used mocha instead). if either of those solutions get added, we could move to using uvu fully (even if its option 1, as ill just write a consumer for it outside)

[43081j]

@lukeed to unblock this and the json issue, would you accept a PR to at least add JSON output for now?

[moltar]

I think having custom reporters, or at least JSON output, opens up a large potential.

uvu sort-a has a small following in serverless architecture testing (see https://dev.to/aws-builders/testing-the-async-cloud-with-aws-cdk-33aj), because of it's light size it can be easily bundled into a Lambda fn.

It would be great to get a machine-readable output, whether it's just JSON or a custom reporter, as this can be then easily logged, or stored in S3, or returned from the test runner Lambda to be saved in the CF output.

[danny-andrews]

Can confirm I'd like to see this. The default reporter is noisy and hard to visually parse IMO.

[SokratisVidros]

+1. The use case is described at #113 (comment)