Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Argument --verbosity is not intuitive #2439

Open
loximann opened this issue Jun 1, 2022 · 4 comments
Open

Argument --verbosity is not intuitive #2439

loximann opened this issue Jun 1, 2022 · 4 comments

Comments

@loximann
Copy link
Contributor

loximann commented Jun 1, 2022

Description
--verbosity in Catch2 applications has a different behaviour than usually expected.

  • --verbose/--verbosity is usually found to increase the output in a normal run for debugging purposes (see eg https://en.wikipedia.org/wiki/Verbose_mode). In Catch2, it is used solely to set the level of detail provided by running with --list-*.
  • The verbosity level "quiet" is not very intuitive: one would expect that normal output is suppressed, eg only errors are printed.
  • The command line help string "set output verbosity" does not indicate that only --list-* runs are affected, and that normal runs are not.

Preferably the option could be renamed to something like "level of detail" (and "quiet" replaced with "low"), as an alias not to break the interface. At least the help string should be changed to something like "set the level of detail for the --list-* options".

Additional context

--list-test-names-only was removed in #1190

@loximann
Copy link
Contributor Author

loximann commented Jun 1, 2022

Preliminary addition to documentation:
https://github.com/loximann/Catch2/blob/document_verbosity/docs/command-line.md#listing-available-tests-tags-or-reporters

I could not document --list-listeners because it crashes. I will look into it tomorrow.

@horenmar
Copy link
Member

horenmar commented Jun 2, 2022

After some investigation, you are right in that verbosity is currently only taken into account by the listings.

However, it is also meant for reporters if they can meaningfully support it. Most of the custom reporters are meant to be machine-friendly, so it is likely meaningless for them, but e.g. the Console reporter could support different verbosities... if someone bothers to add them (I probably won't, as the effort to do so isn't worth it for me).


nitpick on the docs: You should mention that this is for the default listing implementation, and I think that the normal (default) verbosity should be the one with docs, and the other ones should be documented as diffs from the normal. (So normal shouldn't be "same as quiet", but quiet should be "same as normal".)

@loximann
Copy link
Contributor Author

loximann commented Jun 2, 2022

No nitpicking, it is a good suggestion. I have this now:

https://github.com/loximann/Catch2/blob/document_verbosity/docs/command-line.md#listing-available-tests-tags-or-reporters

Do you think I should create a PR and move discussion over there?

@horenmar
Copy link
Member

horenmar commented Jun 2, 2022

Opening up a new PR would be better, yes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants