Skip to content

Check task performance

Jump to bottom
Vyacheslav Rusakov edited this page Feb 17, 2021 · 3 revisions

The following optimization is useful if check task is used often: time-to-time checks without clean.

Check task may take significant time on projects with large classpath. A common example is a gralde plugin project with ~100mb classpath (grale-api.jar).

What animalsniffer do

Animalsniffer signatures (jdk, android) contains classes and methods of all jdk/android classes. But animalsniffer must also know your entire classpath to differentiate 3rd party api usage from missed jdk/android classes or methods.

So for each check run:

  • Signatire file(s) loaded and parsed
  • All classpath jars read (including scanning all jars content to build ignore list)
  • All project files are also added to ignore list

For example, check on gradle plugin could take ~10sec, where actual checking time is only ~1sec and the rest is processing of gradle jars.

Cache

The obvious solution is to load and process all classpath jars once. This way only the first run would be slow and other check runs will be much faster.

The plugin provides special "cache" option for check tasks. To enable cache:

animalsniffer {
    cache.enabled = true
}

When cache enabled, an extra task will be added for each source set (before each animalsniffer check task). This task will build special signature, containing the original signature and all jars in your classpath. This signature will be built once and used for all further check task runs (until you call clean).

Extra tasks are named like: animalsnifferCache[SourceSet name]. For example, main source set tasks pair is animalsnifferCacheMain and animalsnifferMain.

NOTE: For multi-module project, generated signature will not include module jars to avoid signature re-generation because of module changes.

NOTE: When multiple signatures used (jdk and android) for the check, multiple cache signatures will be produced.

Please pay attention that cache will make no sense if you always call animalsniffer after clean or just rarely use it. Cache build is a bit slower then normal check run, so it only makes sense when check used very often. Also, please note that cache has a side effect (read below about exclusions) and known issues (see below).

Reduce cached signature size

NOTE: in most cases, manual optimization is not required. Use it if you need to optimize check task time.

Without the need to load classpath jars, animalsniffer check task time would mainly depends on signature size: signature contains thousands of class descriptors and usually loading signature is slower than overall check. So, to make check task faster, we can reduce generated signature size.

It could be done using special signature info task.

JDK signature

For example, look java6 signature contents:

dependencies {
    signature 'org.codehaus.mojo.signature:java16-sun:1.0@signature'
}

task printSignature(type: ru.vyarus.gradle.plugin.animalsniffer.info.SignatureInfoTask) {
    signature = configurations.signature
    depth = 2
}

It will print:

Signature java16-sun-1.0.signature (1.7 Mb) contains 18312 classes
	com.sun                        7115
	javax.swing                    1781
	sun.awt                        771
	java.util                      662
	sun.nio                        640
	sun.security                   632
	...

Here you can see that java6 signature (1.7mb file) contains 18312 classes. Different depth values could be used to build different views. For example, with depth = 1:

Signature java16-sun-1.0.signature (1.7 Mb) contains 18312 classes
	com                  7115
	sun                  4636
	javax                3327
	java                 2441
	org                  790
	sunw                 3

See that sun.* package contains 4636 classes. These are internal classes and should never be used, so we can safely remove them from the signature. com.sun is also rarely used and could be also excluded. Most likely javax.swing too. So overall we could significantly reduce signature size:

18312 (all) - 4636 (sun) - 7115 (com.sun) - 1781 (javax.swing) = 4780 (more than 3 times smaller!)

Such exclusions are project specific and should be applied manually (if required), but, as sun.* is strongly not recommended for usage, this package is removed by default (more info described below) to use reduced signature out of the box.

Generated source set signature

It is more interesting to analyze signature, generated by cache task (which includes base signature and all jars from the classpath):

dependencies {
    signature 'org.codehaus.mojo.signature:java16-sun:1.0@signature'
    implementation 'junit:junit:4.12'
}

task printSignature(type: ru.vyarus.gradle.plugin.animalsniffer.info.SignatureInfoTask) {
    signature = animalsnifferCacheMain.outputFiles
    depth = 1
}

It will print:

Signature animalsnifferCacheMain.sig (1.3 Mb) contains 14007 classes
	com                  7115
	javax                3327
	java                 2441
	org                  1093
	junit                28
	sunw                 3

Note that generated signature is less than original java signature. This is because sun.* package is removed by default (removing ~4000 classes and reducing the signature size by ~400kb).

This is the default exclusion value:

animalsniffer {
    cache {
        enabled = true
        exclude = ['sun.*', 'org.gradle.internal.impldep.*']
    }
}

org.gradle.internal.impldep.* is repackaged 3rd parties inside gradle jar. This exclusion is very handy for gradle plugin projects (because removes 16239 never used classes from signature).

To exclude other packages use cache.exclude configuration method:

animalsniffer {
    cache {
        enabled = true
        exclude 'com.sun.*', 'javax.swing.*'
    }
}

This will add exclusions without overriding default ones (the method may be called multiple times). Using print task above with this exclusions will give:

Signature animalsnifferResourcesMain.sig (409.4 Kb) contains 5111 classes
	java                 2441
	javax                1546
	org                  1093
	junit                28
	sunw                 3

Overall class count (5111) is larger than it was showed on raw java6 signature because of junit dependency, which classes were also included.

To override default exclusions, assign collection directly to property:

animalsniffer {
    cache {
        enabled = true
        exclude = ['com.sun.*', 'javax.swing.*'] // or simply [] to just remove defaults
    }
}

Known issues

Java 8 signature (org.codehaus.mojo.signature:java18:1.0@signature) may conflict with gradle api:

:animalsnifferCacheMain FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':animalsnifferCacheMain'.
> java.lang.ClassCastException: Cannot merger class javax/xml/xpath/XPathFactoryFinder$1 as it has changed superclass:

There is nothing we can do with it, and for such cases cache can't be used.

Anyway, if you have a problem like this which you can't solve, please create a new issue. Your specific case may help to find some workaround (or even lead to the solution).

Clone this wiki locally