Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JMH benchmark setup #763

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion gradle.properties
Expand Up @@ -3,4 +3,4 @@ org.gradle.configureondemand=false
org.gradle.jvmargs=-XX:MaxMetaspaceSize=768m
localrepo=build/mockk-repo
# localrepo=/Users/raibaz/.m2/repository
# kotlin.version=1.5.10
kotlin.version=1.5.31
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should stay commented (or even better removed), because the kotlin version is being set externally by the github action that runs the builds for several different versions.

It's likely the reason why builds on 1.3.* fail.

29 changes: 29 additions & 0 deletions mockk/performance/README.md
@@ -0,0 +1,29 @@
This is a performance testing setup for `mockk` using [JMH](https://github.com/openjdk/jmh). [kotlinx-benchmark](https://github.com/Kotlin/kotlinx-benchmark) seems to support more than JVM, but I didn't explore other runtimes.

How to run
==========
There are 2 ways to execute the benchmark code

Basic with Gradle
-----------------

```shell
gradle benchmark
```
It will use default configuration set in [build.gradle.kts](./build.gradle.kts) and print
out the results

Advanced with JMH binary
------------------------

If you want access to more JMH feautres you can
```shell
gradle mainBenchmarkJar
```
to build the `.jar` file containing the benchamrk code. It will be saved to `./build/benchmarks/main/jars` directory.

From there you can run JMH with all the JMH-specific configuration options form CLI (even on different machine).
```shell
java -jar ./build/benchmarks/main/jars/mockk-performance-main-jmh-<version>.jar -w 1 -i 2 -r 60s -prof jfr
```
which will e.g. attach the JFR profiler (not possible with `kontlinx-benchmark` executed from Gradle)
32 changes: 32 additions & 0 deletions mockk/performance/build.gradle.kts
@@ -0,0 +1,32 @@
import kotlinx.benchmark.gradle.JvmBenchmarkTarget
import kotlinx.benchmark.gradle.benchmark

plugins {
java
kotlin("jvm")
id("org.jetbrains.kotlinx.benchmark") version "0.4.0"

}

dependencies {
implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.0")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.5.2")
implementation(project(":mockk-jvm"))
}

benchmark {
configurations {
named("main") {
iterationTime = 60
iterationTimeUnit = "sec"
iterations = 2
warmups = 1
}
}
targets {
register("main") {
this as JvmBenchmarkTarget
jmhVersion = "1.33"
}
}
}
23 changes: 23 additions & 0 deletions mockk/performance/src/main/kotlin/io/mockk/performance/JmhTest.kt
@@ -0,0 +1,23 @@
package io.mockk.performance

import io.mockk.every
import io.mockk.mockk
import org.openjdk.jmh.annotations.Benchmark
import org.openjdk.jmh.annotations.BenchmarkMode
import org.openjdk.jmh.annotations.Mode
import org.openjdk.jmh.infra.Blackhole

@BenchmarkMode(Mode.Throughput)
open class JmhTest {

@Benchmark
fun simpleMockAndStub(blackhole: Blackhole) {
val mockedClass: MockedClass = mockk()
every { mockedClass.mockedFun() } returns "Hello, mockk!"
blackhole.consume(mockedClass)
}

class MockedClass {
fun mockedFun(): String = "Hello, world!"
}
}
2 changes: 2 additions & 0 deletions settings.gradle.kts
Expand Up @@ -14,6 +14,7 @@ includeBuild("plugins/configuration")

include("mockk-jvm")
include("mockk-common")
include("mockk-performance")
//include 'mockk-js'

val hasAndroidSdk = extra["hasAndroidSdk"]
Expand Down Expand Up @@ -42,6 +43,7 @@ if (hasAndroidSdk == true) project(":mockk-android").projectDir = file("mockk/an
project(":mockk-agent-api").projectDir = file("agent/api")
project(":mockk-agent-common").projectDir = file("agent/common")
project(":mockk-agent-jvm").projectDir = file("agent/jvm")
project(":mockk-performance").projectDir = file("mockk/performance")
if (hasAndroidSdk == true) {
project(":mockk-agent-android").projectDir = file("agent/android")
project(":mockk-agent-android-dispatcher").projectDir = file("agent/android/dispatcher")
Expand Down