Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce task for JaCoCo plugin for enforcing code coverage metrics #983

Merged
merged 42 commits into from Dec 21, 2016
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
313367a
Basic support for enforcing Jacoco metrics
Dec 2, 2016
d7f3b7d
Add test cases for multiple report tasks
Dec 2, 2016
3f50fcd
Add test case for multiple rules
Dec 2, 2016
5c3a8ad
Break out method
Dec 2, 2016
c27a0b2
Expose configuration option for disabling rules
Dec 2, 2016
3f971f7
Rules can be checked even if all report formats are disabled
Dec 2, 2016
2c7be9f
Use strong typing for enums
Dec 2, 2016
530828f
Rename of method
Dec 2, 2016
493e282
Add some unit tests
Dec 2, 2016
317ec14
Renamed property
Dec 2, 2016
5b0f085
Set default values for fields as described in Jacoco docs
Dec 2, 2016
f8f3070
Renamed fields to better express intention
Dec 2, 2016
9b5ed0a
Rename field
Dec 2, 2016
358a6f9
Enhance samples and documentation
Dec 5, 2016
a73a193
Better IDE support for Closure parameters
Dec 5, 2016
6627b49
Bump up version number
Dec 5, 2016
8e6d20c
Use String data type instead of enum
Dec 5, 2016
70b0f0a
Align terminology with Jacoco configuration options
Dec 5, 2016
aed27f6
Apply input/output annotations
Dec 6, 2016
00b2cb3
Document default value for property
Dec 6, 2016
0dc8b31
Verify proper behavior of Ant task for unknown values
Dec 6, 2016
45674bd
Remove spaces
Dec 6, 2016
ebb4d11
Add multi-version coverage for check support
Dec 6, 2016
41dcf72
Reuse version number across all multi-version tests
Dec 6, 2016
0521afb
Add missing word
Dec 6, 2016
6c63ff9
Use constant for task paths
Dec 6, 2016
db94948
Extract dedicated task for verifying if coverage metrics are met
Dec 6, 2016
f024a94
Input annotation make sure that requires properties are not null
Dec 6, 2016
aa402c8
Reflect use of JacocoCheck task in documentation and sample
Dec 7, 2016
08882e9
Revert change
Dec 7, 2016
6b1f8d9
Use JacocoCheck in documentation
Dec 7, 2016
1eb517c
Reuse task logic for JacocoReport and JacocoCheck
Dec 7, 2016
8d06c8f
Fix indentation
Dec 7, 2016
699a622
Reuse logic
Dec 7, 2016
47feb94
Move logic into parent class
Dec 7, 2016
8097fdd
Task is not cacheable as it doesn't declare outputs
Dec 7, 2016
ff77afc
Ignore running tests on Java 9
Dec 20, 2016
1e31712
Report task should fail if any of the input file does not exist
Dec 20, 2016
a7a4592
Assign group and description for tasks
Dec 20, 2016
d900795
Use task name that better expresses intent
Dec 20, 2016
d832d0d
Change targeted release version
Dec 20, 2016
ec1b003
Fix task name in sample project
Dec 20, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Expand Up @@ -23,7 +23,9 @@ import spock.lang.Unroll
class JacocoPluginCheckCoverageIntegrationTest extends AbstractIntegrationSpec {

private final JavaProjectUnderTest javaProjectUnderTest = new JavaProjectUnderTest(testDirectory)
private final static String[] TEST_AND_JACOCO_REPORT_TASK_PATHS = [':test', ':jacocoTestReport'] as String[]
private final static String[] TEST_TASK_PATH = [':test'] as String[]
private final static String[] JACOCO_REPORT_TASK_PATH = [':jacocoTestReport'] as String[]
private final static String[] TEST_AND_JACOCO_REPORT_TASK_PATHS = TEST_TASK_PATH + JACOCO_REPORT_TASK_PATH
private final static String[] INTEG_TEST_AND_JACOCO_REPORT_TASK_PATHS = [':integrationTest', ':jacocoIntegrationTestReport'] as String[]

def setup() {
Expand Down Expand Up @@ -310,6 +312,43 @@ class JacocoPluginCheckCoverageIntegrationTest extends AbstractIntegrationSpec {
INTEG_TEST_AND_JACOCO_REPORT_TASK_PATHS | 'jacocoIntegrationTestReport' | Limits.Insufficient.CLASS_METRIC_MISSED_COUNT | 'classes missed count is 0.0, but expected minimum is 0.5'
}

def "changes to violation rules re-run task"() {
buildFile << """
jacocoTestReport {
violationRules {
rule {
$Limits.Sufficient.LINE_METRIC_COVERED_RATIO
}
}
}
"""

when:
succeeds TEST_AND_JACOCO_REPORT_TASK_PATHS

then:
executedAndNotSkipped(TEST_AND_JACOCO_REPORT_TASK_PATHS)

when:
succeeds TEST_AND_JACOCO_REPORT_TASK_PATHS

then:
executed(TEST_AND_JACOCO_REPORT_TASK_PATHS)
skipped(TEST_AND_JACOCO_REPORT_TASK_PATHS)

when:
buildFile << """
jacocoTestReport.violationRules.rules[0].limits[0].maximum = 0.5
"""

fails TEST_AND_JACOCO_REPORT_TASK_PATHS

then:
executed(JACOCO_REPORT_TASK_PATH)
skipped(TEST_TASK_PATH)
errorOutput.contains("Rule violated for bundle $testDirectory.name: lines covered ratio is 1.0, but expected maximum is 0.5")
}

static class Limits {
static class Sufficient {
static final String LINE_METRIC_COVERED_RATIO = Limits.create('LINE', 'COVEREDRATIO', '0.0', '1.0')
Expand Down
Expand Up @@ -17,20 +17,25 @@
package org.gradle.testing.jacoco.tasks.rules;

import org.gradle.api.Incubating;
import org.gradle.api.tasks.Input;
import org.gradle.api.tasks.Optional;

import java.io.Serializable;

/**
* Defines a Jacoco rule limit.
*
* @since 4.0
*/
@Incubating
public interface JacocoLimit {
public interface JacocoLimit extends Serializable {

/**
* The counter that applies to the limit as defined by
* <a href="http://www.eclemma.org/jacoco/trunk/doc/api/org/jacoco/core/analysis/ICoverageNode.CounterEntity.html">org.jacoco.core.analysis.ICoverageNode.CounterEntity</a>.
* Valid values are INSTRUCTION, LINE, BRANCH, COMPLEXITY, METHOD and CLASS. Defaults to INSTRUCTION.
*/
@Input
String getCounter();

void setCounter(String counter);
Expand All @@ -40,20 +45,25 @@ public interface JacocoLimit {
* <a href="http://www.eclemma.org/jacoco/trunk/doc/api/org/jacoco/core/analysis/ICounter.CounterValue.html">org.jacoco.core.analysis.ICounter.CounterValue</a>.
* Valid values are TOTALCOUNT, MISSEDCOUNT, COVEREDCOUNT, MISSEDRATIO and COVEREDRATIO. Defaults to COVEREDRATIO.
*/
@Input
String getValue();

void setValue(String value);

/**
* Gets the minimum expected value for limit. Default to null.
*/
@Input
@Optional
Double getMinimum();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would make more sense for these to be integers and explain what they mean based on the context of whichever element they're in.

There are 5 possible values, 3 are "counts" (so must be integers) and 2 are ratios (which could be doubles). We could just as easily represent a ratio of 1.0 as a percentage of 100%.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am going to use BigDecimal in accordance to Maven JaCoCo plugin.


void setMinimum(Double minimum);

/**
* Gets the maximum expected value for limit. Default to null.
*/
@Input
@Optional
Double getMaximum();

void setMaximum(Double maximum);
Expand Down
Expand Up @@ -20,7 +20,10 @@
import groovy.lang.DelegatesTo;
import org.gradle.api.Action;
import org.gradle.api.Incubating;
import org.gradle.api.tasks.Input;
import org.gradle.api.tasks.Nested;

import java.io.Serializable;
import java.util.List;

import static groovy.lang.Closure.DELEGATE_FIRST;
Expand All @@ -31,13 +34,14 @@
* @since 4.0
*/
@Incubating
public interface JacocoViolationRule {
public interface JacocoViolationRule extends Serializable {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Silly question: why does this need to be Serializable?


void setEnabled(boolean enabled);

/**
* Indicates if the rule should be used when checking generated coverage metrics.
*/
@Input
boolean isEnabled();

void setElement(String element);
Expand All @@ -47,6 +51,7 @@ public interface JacocoViolationRule {
* <a href="http://www.eclemma.org/jacoco/trunk/doc/api/org/jacoco/core/analysis/ICoverageNode.ElementType.html">org.jacoco.core.analysis.ICoverageNode.ElementType</a>.
* Valid scope values are BUNDLE, PACKAGE, CLASS, SOURCEFILE and METHOD. Defaults to BUNDLE.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any kind of precedence here? e.g., if I define a limit for PACKAGE and another limit for CLASS, does one override the other?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about that. IMHO that's something the Ant task should handle for us.

*/
@Input
String getElement();

void setIncludes(List<String> includes);
Expand All @@ -55,6 +60,7 @@ public interface JacocoViolationRule {
* List of elements that should be included in check. Names can use wildcards (* and ?).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a Ant-like pattern or something different?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has been taken from the JaCoCo documentation (see "Element check").

* If left empty, all elements will be included. Defaults to [*].
*/
@Input
List<String> getIncludes();

void setExcludes(List<String> excludes);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need javadoc on the setters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expand All @@ -63,11 +69,13 @@ public interface JacocoViolationRule {
* List of elements that should be excluded from check. Names can use wildcards (* and ?).
* If left empty, no elements will be excluded. Defaults to an empty list.
*/
@Input
List<String> getExcludes();

/**
* Gets all limits defined for this rule. Defaults to an empty list.
*/
@Nested
List<JacocoLimit> getLimits();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this works (but I would need to check...). I think we would end up looking for annotations on List.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Will use @Input instead.


/**
Expand Down
Expand Up @@ -20,7 +20,7 @@
import groovy.lang.DelegatesTo;
import org.gradle.api.Action;
import org.gradle.api.Incubating;
import org.gradle.api.tasks.Internal;
import org.gradle.api.tasks.Input;

import java.util.List;

Expand All @@ -39,13 +39,13 @@ public interface JacocoViolationRulesContainer {
/**
* Specifies whether build should fail in case of rule violations. Defaults to true.
*/
@Internal
@Input
boolean isFailOnViolation();

/**
* Gets all violation rules. Defaults to an empty list.
*/
@Internal
@Input
List<JacocoViolationRule> getRules();

/**
Expand Down