Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect code coverage metadata does not prevent code coverage data from being collected #5351

Closed
jrfnl opened this issue May 2, 2023 · 1 comment
Assignees
Labels
event/code-sprint/2024-03 PHPUnit Code Sprint: March 2024 feature/code-coverage Issues related to code coverage (but not php-code-coverage) feature/metadata Issues related to attributes and annotations type/bug Something is broken

Comments

@jrfnl
Copy link
Contributor

jrfnl commented May 2, 2023

Q A
php-code-coverage version 10.1.1
PHP version 8.2.5
Driver Xdebug
Xdebug version (if used) 3.2.1
Installation Method irrelevant, same result in both cases
Usage Method PHPUnit
PHPUnit version (if used) 10.1.2

Given

  • A phpunit.xml.dist config file which includes the forceCoversAnnotation="true" (PHPUnit 9.x) or requireCoverageMetadata="true" (PHPUnit 10.x) directive;
  • And a test which has a @covers tag for a method which doesn't exist.

I'd expect

That no code coverage would be recorded for the test with the incorrect @covers tag, along the same lines as if the test had a @covers tag for a method which wasn't run during the test.

And if the test had multiple @covers tags, I'd expect code coverage only to be recorded for the valid tags.

What happened instead

Coverage was recorded as if forceCoversAnnotation/requireCoverageMetadata was set to false, i.e. all code which was run during the test now shows as covered, even though there are no tests with the appropriate @covers tags for that code.

Reproduction scenario

See the individual commits in the https://github.com/jrfnl/bug-report-reproduction-scenarios/tree/php-code-coverage/997-wrong-covers-tag-wrong-coverage branch.
Each commit includes the coverage reports (text summary in the commit message, HTML + Clover reports in the actual commits).

@jrfnl jrfnl changed the title Incorrect @covers tag causes coverage to be recorded while forceCoversAnnotation is on Incorrect @covers tag causes coverage to be recorded while requireCoverageMetadata is on May 2, 2023
@sebastianbergmann sebastianbergmann added the type/bug Something is broken label May 3, 2023
@sebastianbergmann sebastianbergmann transferred this issue from sebastianbergmann/php-code-coverage May 3, 2023
@sebastianbergmann sebastianbergmann added the feature/code-coverage Issues related to code coverage (but not php-code-coverage) label May 3, 2023
@sebastianbergmann sebastianbergmann changed the title Incorrect @covers tag causes coverage to be recorded while requireCoverageMetadata is on Incorrect code coverage metadata does not prevent code coverage data from being collected Mar 9, 2024
@sebastianbergmann sebastianbergmann added event/code-sprint/2024-03 PHPUnit Code Sprint: March 2024 feature/metadata Issues related to attributes and annotations labels Mar 9, 2024
@sebastianbergmann sebastianbergmann self-assigned this Mar 9, 2024
@jrfnl
Copy link
Contributor Author

jrfnl commented Mar 9, 2024

Thank you @sebastianbergmann ! Happy to see this fix will be in the next releases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
event/code-sprint/2024-03 PHPUnit Code Sprint: March 2024 feature/code-coverage Issues related to code coverage (but not php-code-coverage) feature/metadata Issues related to attributes and annotations type/bug Something is broken
Projects
None yet
Development

No branches or pull requests

2 participants