Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exit code goes against Unix conventions - breaks basic assumptions #1410

Closed
jamesfrullo opened this issue Oct 16, 2018 · 1 comment
Closed

Exit code goes against Unix conventions - breaks basic assumptions #1410

jamesfrullo opened this issue Oct 16, 2018 · 1 comment
Labels
Duplicate
Milestone
3.0

Comments

@jamesfrullo
Copy link

jamesfrullo commented Oct 16, 2018 •
edited

Description

When your run your unit test with "-l" or "--list-test-names-only", the exit code of the process is the number of tests. This is quite incorrect behavior on any Unix operating system. A non-zero exit code indicates the program could not perform its intended function. It is reasonable for a tool that runs the program to assume that something went wrong with listing tests, and indeed, I'm using Catch in a framework that assumes exactly that. Furthermore, exit codes above 128 are supposed to have special meaning, and the exit code may not exceed 255. See here, for example:
https://www.tldp.org/LDP/abs/html/exit-status.html

Please fix this to have the test runner exit 0 if there is, in fact, no error.

Steps to reproduce

Simply build and run any catch unit test with "-l" or "--list-test-names-only".

Extra information

  • Catch version: v2.4.1
  • Operating System: Linux
  • Compiler+version: GCC 5.4
@horenmar
Copy link
Member

#1146

The TL;DR version is that it won't be changed until next major version.

@onqtam onqtam mentioned this issue Jan 10, 2019
Closed
@JoeyGrajciar JoeyGrajciar added this to the 3.0 milestone Jan 31, 2019
horenmar added a commit that referenced this issue Jun 16, 2019
@horenmar
Make --list-* exit code be 0
fcc4db1 
Previously it returned the sum of listed things because ???. This
was completely useless and in many ways actively counterproductive
because of the success/failure conventions around exit codes.

Closes #1410
@horenmar horenmar closed this as completed Oct 4, 2019
horenmar added a commit that referenced this issue Oct 28, 2019
@horenmar
Make --list-* exit code be 0
2c06ee9 
Previously it returned the sum of listed things because ???. This
was completely useless and in many ways actively counterproductive
because of the success/failure conventions around exit codes.

Closes #1410
horenmar added a commit that referenced this issue Oct 29, 2019
@horenmar
Make --list-* exit code be 0
d2cddfc 
Previously it returned the sum of listed things because ???. This
was completely useless and in many ways actively counterproductive
because of the success/failure conventions around exit codes.

Closes #1410
@lkeegan lkeegan mentioned this issue Oct 8, 2020
Merged
lkeegan added a commit to lkeegan/Catch2 that referenced this issue Oct 8, 2020
@lkeegan
fix warning in extras/CatchAddTests.cmake
c44d9cc 
  - don't warn on zero return code of --list-reporters
  - previously return code was the number of reporters (catchorg#1410, catchorg#1146)
  - as of 2c06ee9 return code is zero on success
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Duplicate
Projects
None yet
Milestone
3.0
Development

No branches or pull requests
3 participants
@JoeyGrajciar @jamesfrullo @horenmar