Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow multiple reporters with multiple output files #1712

Closed
horenmar opened this issue Aug 2, 2019 · 3 comments
Closed

Allow multiple reporters with multiple output files #1712

horenmar opened this issue Aug 2, 2019 · 3 comments
Assignees
Milestone

Comments

@horenmar
Copy link
Member

horenmar commented Aug 2, 2019

Description

There have been requests for specifying multiple reporters to be used for the same test run. This was implemented for a while, but it ran into trouble caused by the fact that there was no way to give them different output files/streams for them to write to. If we reintroduce this possibility, we also need to decide on a syntax that allows different reporters to write to different files/streams.

Additional context
Possible syntaxes:

  • -r takes a list, -o takes a list, the size of list in -r must be at most 1 longer than the one in -o. Reporters are then given the targets from -o in the same order they are specified.
    Examples: -r junit xml -o junit.out -> junit reporter writes to the file junit.out, xml reporter writes to stdout.
  • -r takes a list of reporters, where the name can be followed by a colon and the name of the output file, e.g. -r junit:junit.out would say that the junit reporter should write its output to junit.out. Stdout would then be specified as either -, or without any followup after the name.

We also need to decide what to do if multiple reporters would be writing to stdout -- ignore it? Warn the user? Error out?

@horenmar horenmar added this to the 3.0 milestone Aug 2, 2019
@JoeyGrajciar
Copy link
Contributor

Personally i would go with second option.
It would be less prone to errors in defining of the reporters order.
Stdout could be specified by leaving out the optional :<filename> part.

I would go with error for mutliple reporters writing to stdout.

@JoeyGrajciar JoeyGrajciar self-assigned this Oct 30, 2019
@macetw
Copy link

macetw commented Dec 3, 2020

This is a basic fundamental need. CI frameworks (like Jenkins) need this information. Developers also need this information while running on their console, and they're not interested in digging in to XML files fishing out this information. It's also important that the same test logic runs on development workstations as what runs on CI frameworks. Please prioritize this request.

@WimLeflere
Copy link

@macetw with the current reporters and output file options you can already have different output for CI and developers.
On CI you can for example use the xml reporter and write it to a file.
On the developers machine you can use the console reporter and write to the terminal.
With the same test code/executable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants