Project for benchmarking popular Json Web Token (JWT) frameworks for Java using JMH.
Supported frameworks:
- java-jwt from Auth0
- jjwt from jwtk
- Okta JWT Verifier for Java from Okta
- fusionauth-jwt from FusionAuth
- Nimbus-JOSE-JWT from connect2id
The project is based on Gradle.
Modify the build version to your current snapshot, then run
./gradlew --stop && ./gradlew clean jmhClasses jmh --refresh-dependencies --info
Note: Requires Java 8 to build, there is some classpath issue with Java 11 which I've yet to understand.
Temporarily disable boost (untill next reboot) for AMD:
echo "0" | sudo tee /sys/devices/system/cpu/cpufreq/boost
The JMH plugin seems to have trouble refreshing the project, so restart the Gradle deamon before running.
- Parse token (if possible)
- Verify token
- Verify token, get claim.
There is not much headroom for improving the performance of RSA-based signatures; most (> 90%) of the CPU time goes to signature verification.
This observation is based on two simple 'baseline' RSA-based JWT verifiers included in this project. They perform about 5% faster than the fastest JWT libraries, but then only does signature verification and no JSON parsing / field validation. So it seems likely this is the upper limit.
Running for OpenJDK 21 build 21.0.6+7 on Fedora Linux 6.12.9-200, AMD Ryzen 9950.
Result visualizations:
Summary: For JWT verification fusionauth-jwt and java-jwt are tied for first place. fusionauth-jwt can parse faster (without verification).
Note that the relative number matters, benchmarks are not necessarily run on the same machine, operating system or Java version.
Running for OpenJDK 11 build 11.0.18+10 and OpenJDK 17 build 17.0.6+10, on Fedora Linux 6.1.8-200, AMD Ryzen 5950.
- [visualization_v1.5.0]
Summary: For JWT verification fusionauth-jwt and java-jwt are tied for first place. fusionauth-jwt can parse faster (without verification).
Running for OpenJDK 11 build 11.0.11.0.9, Fedora Linux 5.13.4-200, AMD Ryzen 7 PRO 3700U laptop.
A visualization_v1.4.0 is available. In short, fusionauth-jwt is the fastest parser, but Java-jwt is very close.
Running for OpenJDK 1.8.0 build 275-b01, Fedora Linux 5.10.7-200, AMD 5950x.
A visualization_v1.3.0 is available, summery:
| Framework | Version | Verify (op/s) | Claim (op/s) | Parse (op/s) |
|---|---|---|---|---|
| java-jwt | 3.12.0/0.15.0 | 30.1k | 29.9k | 387k |
| fusionauth-jwt | 4.0.1 | 32.6k | 32.4k | 1196k |
| jjwt | 0.9.1 | 31.4k | 30.7k | |
| Nimbus-JOSE-JWT | 9.4.1 | 19.7k | 18.9k | 945k |
| Okta JWT Verifier for Java | 0.5.0 | 1.3k | 1.3k |
In short, fusionauth-jwt is the fastest parser. Parsing without validation is now faster than before for java-jwt, otherwise no improvements.
Running for OpenJDK 1.8.0 build 232, Linux 5.3.7.
A visualization_v1.2.0 is available, summery:
| Framework | Version | Verify (op/s) | Claim (op/s) | Parse (op/s) |
|---|---|---|---|---|
| java-jwt | 3.8.3/0.9.0 | 14.6k | 14.6k | 230.8k |
| jjwt | 0.9.1 | 14.9k | 14.5k | |
| Okta JWT Verifier for Java | 0.4.0 | 0.6k | 0.6k | |
| fusionauth-jwt | 3.1.6 | 15.9k | 15.7k | 581.5k |
In short, fusionauth-jwt is the new fastest parser. Parsing without validation is now faster than before for java-jwt, otherwise no improvements.
Running for OpenJDK 1.8.0 build 191.
A visualization_v1.1.0 is available, summery:
| Framework | Version | Verify (op/s) | Claim (op/s) |
|---|---|---|---|
| java-jwt | 3.7.0/0.7.0 | 14.5k | 14.6k |
| jjwt | 0.9.1 | 14.7k | 14.9k |
| Okta JWT Verifier for Java | 0.4.0 | 0.61k | 0.61k |
In short, java-jwt did catch up with jjwt, they were essentially just as fast. The Okta JWT verifiser regressed into a total disaster.
Only java-jwt seems to parse without validation, at about 52.3k operations per second.
Running for Oracle JDK 1.8 build unknown-
A visualization_v1.0.0 is available, summery:
| Framework | Version | Verify (op/s) | Claim (op/s) |
|---|---|---|---|
| java-jwt | 3.3.0/0.4.0 | 14.2k | 13.9k |
| jjwt | 0.9.0 | 19k | 18.3k |
| Okta JWT Verifier for Java | 0.3.0 | 17.1k | 17.1k |
Only java-jwt seems to parse without validation, at about 65.2k operations per second.
- 1.6.0: Updated dependencies
- 1.x: Added baseline JWT verifiers
- 1.5.0: Updated dependencies
- 1.3.0: Updated dependencies, added Nimbus JOSE + JWT
- 1.2.0: Added FusionAuth
- 1.1.0: Bumped versions after accepted performance-enhancing PR#255 for java-jwt.
- 1.0.0: Initial version
![@renovate[bot]](https://avatars.githubusercontent.com/in/2740?s=64&v=4){kind=small}
