[jvm] Scan classes in parallel #905
Conversation
| @@ -44,13 +48,15 @@ object JvmDiscoveryContextFactory { | |||
| private fun scanClasses(testDirs: List<String>): List<KClass<out Spek>> { | |||
| val cg = ClassGraph() | |||
| .enableClassInfo() | |||
| // any jar on the classpath won't be scanned | |||
| .disableJarScanning() | |||
|
@arturbosch new build is out https://bintray.com/spekframework/spek-dev/spek2/2.0.13-alpha.0.1%2Bcecb731 can you try it out? |
I've updated detekt/detekt#2902 (comment) This patch decreases the discovery time by 1 second 🎉 . Jupiter still is 1 second faster though. |
|
Ohh, that's good news! I'll see if I can configure classgraph further to scan less. |
|
Our full test time without the gradle plugin decreased by ~ 20s from 2m8s to 1m47s! Is Spek using a lot of concurrency by default? If yes, maybe this could lead to threads starving when used together with I see 10seconds discovery time for very simple modules: With snapshot: With 2.0.12: |
| @@ -12,6 +12,10 @@ import kotlin.reflect.full.primaryConstructor | |||
| import kotlin.streams.toList | |||
|
|
|||
| object JvmDiscoveryContextFactory { | |||
| private val scanParallelism by lazy { | |||
| Runtime.getRuntime().availableProcessors() | |||
There was a problem hiding this comment.
Lazy shouldn't be necessary here?
If I use gradle --parallel with 8 cores this will start 8*8 threads to scan the classpath?
There was a problem hiding this comment.
I was about to link this, and yeah it probably will. I'm still thinking of a way to make this configurable.
There was a problem hiding this comment.
Maybe via a jvm property? https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html#org.gradle.api.tasks.testing.Test:jvmArgs WDYT?
There was a problem hiding this comment.
It sucks that you can't pass configs from gradle to JUnit Platform test engines.
There was a problem hiding this comment.
I think this should be sequential by default and be configurable like you did with the timeout property:
systemProperty("SPEK_TIMEOUT", 0) // disable test timeout.
Junit5 itself wants to be configured for concurrency with system properties: https://junit.org/junit5/docs/current/user-guide/#running-tests-build-gradle-config-params
There was a problem hiding this comment.
Ahh that makes sense - I don't have time now, but I'll look into it in the weekend. Thanks!
|
I had a closer look, |
(cherry picked from commit cecb731)
//cc @arturbosch
I'll merge this to the
2.0.xbranch which will trigger a CI build and publish an artifact to: https://bintray.com/spekframework/spek-dev/