Plans for Dropwizard 3

[zUniQueX]

Including Logback's alpha versions, the dependencies could now be updated to the jakarta namespace.

Should we - as most other Jakarta EE components - support both the javax and the jakarta namespace and therefore build Dropwizard 3, or should we postpone the upgrade?

[zUniQueX]

At the moment, the upgrade seems to get really heavy, since Jetty is on version 9 yet. Unfortunately we cannot upgrade Jetty to 10.0.x due to a logback issue. A PR for this is already raised: qos-ch/logback#532

[zUniQueX]

@joschi For an easier upgrade to the jakarta namespace I took a look at an upgrade of Jetty to 10.x.x. This commit does the job, however I'm not quite pleased with it. Logback hereby is the current blocking factor, since the JettyServerAdapter does not support HttpFields.Mutable. I introduced a temporary custom Jetty10ServerAdapter, but therefore also had to implement the AppenderAttachableImplExposingRequestLogImpl, which makes me unhappy about this.
Since I don't know, when the referenced PR will get merged, I see this as an alternative anyway, if we would want to do the big bang change.

Should I provide a PR for this? The newly introduced classes are marked as deprecated and will be removed when using the Logback classes that support the jakarta namespace. I just think, that without the Jetty upgrade the major upgrade of Dropwizard to v3 will get quite messy.

[dsvensson]

Any progress for this? Just bumped to latest dropwizard 3.x and I'm getting a method not found on that one:

java.lang.NoSuchMethodError: 'org.eclipse.jetty.http.HttpFields org.eclipse.jetty.server.Response.getHttpFields()'
	at ch.qos.logback.access.jetty.JettyModernServerAdapter.buildResponseHeaderMap(JettyModernServerAdapter.java:57)
	at ch.qos.logback.access.spi.AccessEvent.buildResponseHeaderMap(AccessEvent.java:599)
	at ch.qos.logback.access.spi.AccessEvent.getResponseHeaderMap(AccessEvent.java:604)
	at ch.qos.logback.access.spi.AccessEvent.prepareForDeferredProcessing(AccessEvent.java:616)
	at io.dropwizard.request.logging.async.AsyncAccessEventAppenderFactory$1.preprocess(AsyncAccessEventAppenderFactory.java:22)
	at io.dropwizard.request.logging.async.AsyncAccessEventAppenderFactory$1.preprocess(AsyncAccessEventAppenderFactory.java:19)
	at ch.qos.logback.core.AsyncAppenderBase.append(AsyncAppenderBase.java:165)
	at ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:85)
	at com.foo.AccessWithContextRequestLog$log$1.invoke(AccessWithContextRequestLogFactory.kt:67)
	at com.foo.AccessWithContextRequestLog$log$1.invoke(AccessWithContextRequestLogFactory.kt:67)
	at com.foo.utils.AccessWithContextRequestLog.log$lambda$0(AccessWithContextRequestLogFactory.kt:67)
	at java.base/java.util.concurrent.CopyOnWriteArrayList$COWIterator.forEachRemaining(CopyOnWriteArrayList.java:1152)
	at com.foo.AccessWithContextRequestLog.log(AccessWithContextRequestLogFactory.kt:67)

Where the implementation looks like the following:

class AccessWithContextRequestLog : LogbackAccessRequestLog() {
	override fun log(jettyRequest: Request?, jettyResponse: Response?) {
		val adapter = JettyModernServerAdapter(jettyRequest, jettyResponse)

		val accessEvent: IAccessEvent = AccessWithContextEvent(this, jettyRequest, jettyResponse, adapter)

		MDC.clear() // Clearing MDC for next requests

		if (getFilterChainDecision(accessEvent) === FilterReply.DENY) {
			return
		}
		iteratorForAppenders().forEachRemaining { appender -> appender.doAppend(accessEvent) }
	}
}

class AccessWithContextEvent(ctx: Context, httpRequest: HttpServletRequest?, httpResponse: HttpServletResponse?, adapter: ServerAdapter?) : AccessEvent(ctx, httpRequest, httpResponse, adapter) {
	val mdc: Map<String, String> = MDC.getCopyOfContextMap() ?: mapOf()
}

I thought that was what using JettyModernServerAdapter should have helped with as suggested in #6936

[zUniQueX]

@dsvensson Logback is broken for Jetty 10. Although the JettyModernServerAdapter was introduced to work with more modern Jetty versions, Logback didn't upgrade to Jetty 10.

Logback 1.3.x still builds with Jetty 9.4. The return type of getHttpFields() has changed in Jetty 10, but as Logback 1.3 still got the JAR for Jetty 9, the new type isn't present there. I've proposed a change for that some time ago, but my PR isn't merged yet.

We work around this problem ourselves with a custom ServerAdapter implementation. However, this class is package-private, so you'll have to copy it to your project in order to use it.

[dsvensson]

Thank you so much, saved me a bunch of time.

[dsvensson]

That was the last piece. Bumped to 3.x now. Just a tsunami of Jakarta-work for other deps until 4.x.

[pstackle]

I wonder if we should be considering an approach similar to what Jetty did with their v10 and v11 split. We would do Dropwizard v3 synced up with Jetty 10 and stay on the javax namespace and simultaneously create v4 that syncs up with Jetty 11 and uses the jakarta namespace.

Ref: https://webtide.com/jetty-10-and-11-have-arrived/

CC @dropwizard/committers

[zUniQueX]

I wonder if we should be considering an approach similar to what Jetty did with their v10 and v11 split. We would do Dropwizard v3 synced up with Jetty 10 and stay on the javax namespace and simultaneously create v4 that syncs up with Jetty 11 and uses the jakarta namespace.

Ref: https://webtide.com/jetty-10-and-11-have-arrived/

CC @dropwizard/committers

@pstackle I think this would be nice for users but will be very hard to maintain. Keeping only two versions in sync with the javax and jakarta namespaces gets a difficult task. And with this configuration we would have to port changes also for the three different Jetty versions.

[pstackle]

@zUniQueX What do you mean by three Jetty versions?

I was thinking we'd have:

• Dropwizard v3 (targeted at Jetty 10 supporting the javax namespace)
• Dropwizard v4 (targeted at Jetty 11 supporting the jakarta namespace)

with Dropwizard v2 (and Jetty 9) no longer supported or at least in a (hopefully) low-effort maintenance mode.

---

Alternatively, if we don't mirror the approach Jetty has done, we'd basically have to pick between the following choices:

• Dropwizard v3 targets Jetty 10, continues using javax namespace and users have to wait longer to complete the javax to jakarta migration until some time in the future when a new Dropwizard version targets Jetty 11+
• Dropwizard v3 targets Jetty 11 (skips Jetty 10), users have a hard cutover from (Jetty 9 + javax) to (Jetty 11 + jakarta)

[zUniQueX]

@pstackle I exactly meant Jetty 9 in v2, Jetty 10 in v3 and Jetty 11 in v4. I think, we cannot directly deprecate v2.

In my opinion we can skip a Jetty 10 only version, so I would stay on Jetty 9 in v2 and upgrade to Jetty 11 in v3.

[joschi]

@dropwizard/committers @zUniQueX What do you think about the following roadmap?

• Dropwizard 2.2.x with Jetty 10.x (and Java 11 baseline)
• Dropwizard 3.x with Jetty 11.x (and Java 11 baseline and Jakarta EE namespace)

While Dropwizard 2.2.x formally is breaking semantic versioning (because of the change of the language baseline from Java 8 to 11), it would still work largely as before and it might give users a little nudge to upgrade to Java 11 in production. 😉

Other than that I don't think Jetty 10 will break a lot of things which are officially supported by Dropwizard (see also https://webtide.com/jetty-10-and-11-have-arrived/) and thus the upgrade from Dropwizard 2.1.x to 2.2.x should cause only minor upgrade pain - if at all when people are already on Java 11.

[danjee]

Hello, is there a dropwizard 3 preview repository available? I am only interested in the jakarta libs upgrade. I've downloaded and tried to build the project on the renovate-jakarta branch but got some errors there, I've updated the jersey libs but still got into some problems with jackson which uses old javax packages. They had a bom with jakarta classifierd but revert it. Looks like it's a long way to have it.

[zUniQueX]

If you just want to test something with the libraries updated to their jakarta versions, you can check out this one: #4727

[joschi]

@danjee No, this is currently only in planning stage and there's no development branch yet.

[danjee]

@joschi thanks. I've edited the question

[joschi]

@danjee Moving from javax.* to jakarta.* is a huge effort and we need to upgrade basically all libraries which are using javax.* right now.

This is not only Jersey and Jetty, but also Hibernate Validator, Hibernate ORM, and any library adding functionality to those aforementioned ones.

In other words: Don't hold your breath. 😉

[mattnelson]

• Dropwizard 2.2.x with Jetty 10.x (and Java 11 baseline)
• Dropwizard 3.x with Jetty 11.x (and Java 11 baseline and Jakarta EE namespace)

While Dropwizard 2.2.x formally is breaking semantic versioning (because of the change of the language baseline from Java 8 to 11), it would still work largely as before and it might give users a little nudge to upgrade to Java 11 in production.

Would request documented timelines on end of support for 2.x, 2.1.x, and 2.2.x so consumers can start planning around it as soon as possible.

[joschi]

Let's concentrate on bringing Dropwizard 2.1.0 out the door as soon as possible so that people can upgrade and see what will be deprecated in Dropwizard 3.0.x and 4.0.x.

Dropwizard 3.0.x can then target the Java 11+ crowd and finally Dropwizard 4.0.x the Jakarta EE crowd.

[zUniQueX]

👍 is anything else required for 2.1.0 without an open PR? Then I'd like to take a look at that.

[wardev]

First, thanks for making Dropwizard and having this discussion!

In many cases Java 8 is supported longer than Java 11. E.g. Eclipse's build of OpenJDK 8 is supported through 2026 while 11 is only supported through 2024. [1] Similar for RedHat [2]. For Oracle the dates are 2030 and 2026, respectively [3]. So it seems Java 8 has better support going forward than Java 11.

As a user I would like stability in the platforms/frameworks I use. Therefore I would like to see Java 8 supported through at least 2026.

The reason is that since Oracle decided that Java releases should no longer be backwards compatible it creates a significant amount of make-work for developers to update Java versions with little to no added value for the existing code base. Supporting Java 8 through 2026 will make it possible for users to migrate to whichever version of Java vendors decide to support after 2030. (I.e. users won't have to migrate to a version of Java with shorter support than Java 8.)

Regards,
Evan

[1] https://adoptium.net/support/
[2] https://access.redhat.com/articles/1299013
[3] https://en.wikipedia.org/wiki/Java_version_history

[zUniQueX]

We probably won't support Dropwizard 2.x that long. Previously, the last version got security updates while the current version has full support.

As we now upgraded to Jetty 10 and 11 with Dropwizard 3.x and 4.x respectively, we will have to maintain at least two major versions. Dropwizard 2.x will get support for some time, but we would encourage users to use the jakarta libraries (4.x onwards), as the library authors focus on those versions.

[joschi]

As a user I would like stability in the platforms/frameworks I use. Therefore I would like to see Java 8 supported through at least 2026.

Would you be willing to take over maintenance of the Dropwizard 2.x branches which are currently targeting Java 8 as their language baseline?

IMHO, the long support for Java 8 is a big issue and is holding back the Java ecosystem in a very harmful way. YMMV.

[zUniQueX]

@dropwizard/committers @zUniQueX What do you think about the following roadmap?

• Dropwizard 2.2.x with Jetty 10.x (and Java 11 baseline)
• Dropwizard 3.x with Jetty 11.x (and Java 11 baseline and Jakarta EE namespace)

While Dropwizard 2.2.x formally is breaking semantic versioning (because of the change of the language baseline from Java 8 to 11), it would still work largely as before and it might give users a little nudge to upgrade to Java 11 in production. 😉

@joschi From other PRs and issues I thought you would highly prefer semantic versioning and therefore wouldn't want to make such an upgrade in 2.2.x 😉 I personally agree Java 11 upgrades should be done for production environments and therefore would appreciate the change.

For the upgrade effort I previously linked a test branch I have for this.
Currently I see these points to note for an upgrade:

• addEventListener: The upgrade to the addEventListener method should be not much a problem, as the compiler would complain about this.
• GZIP User-Agent exclusion: As exclusion of User-Agents got removed, the property in the configuration is just unnecessary
• GZIP default accepted methods: from Jetty 10 onwards, GET and POST are enabled by default. This also shouldn't be a suprising change to users, but could be nice to note.
• Additional bean in LifecycleEnvironment: LifeCycleEnvironment now doesn't only contain the JettyManaged objects.
• SNI host checking: Jetty 10 uses are more strict implementation of server-side SNI host checking in the SecureRequestsCustomizer. Therefore previously valid client configurations will now fail. But I didn't want to set the disableSniHostCheck property in the default configuration.
• shutdownGracePeriod in HTTP/2 tests: I couldn't find out why the HTTP/2 integration tests would only fail sometimes in various executions. In the logs can be seen, that the request was already done, when an exception occurred. Disabling graceful shutdown seems to fix the problem, so I see this as a reasonable workaround for now. If any user does a similar thing, this could be very suprising.

Currently the limiting factor of doing an upgrade seems to be that Logback still doesn't support Jetty 10. We could work around this with a custom implementation of the JettyServerAdapter, but I don't know, if we should do this.

[zUniQueX]

Hello, is there a dropwizard 3 preview repository available? I am only interested in the jakarta libs upgrade. I've downloaded and tried to build the project on the renovate-jakarta branch but got some errors there, I've updated the jersey libs but still got into some problems with jackson which uses old javax packages. They had a bom with jakarta classifierd but revert it. Looks like it's a long way to have it.

IIRC the only necessary Jackson dependency for this upgrade has a Jakarta version with the jackson-jakarta-rs-json-provider.

[zUniQueX]

@danjee Moving from javax.* to jakarta.* is a huge effort and we need to upgrade basically all libraries which are using javax.* right now.

This is not only Jersey and Jetty, but also Hibernate Validator, Hibernate ORM, and any library adding functionality to those aforementioned ones.

In other words: Don't hold your breath. 😉

@joschi Would you be interested in having an early development branch for DW 3? After the Jetty 10 upgrade this is more or less replacing libraries and doing search-and-replace in the IDE 😉 I've done this before and could complete the mvn install. For more I just didn't find the time. I could upgrade this branch to the current master and provide a PR, if there would be interest in this.

[joschi]

Would you be interested in having an early development branch for DW 3?

I've just created the release/3.0.x (for Jetty 10, Java 11) and release/4.0.x (for Jakarta EE) branches against which we can create pull requests.

Please go crazy. 😉

[zUniQueX]

Sounds great 👍 I'll then provide a PR for the Jakarta EE upgrade. I just searched for the latest dependency versions and noticed Jetty 11.0.8 broke something at request logging, so I will ship the branch with 11.0.7.

[pstackle]

Dropwizard 2.2.x with Jetty 10.x (and Java 11 baseline)

@joschi Why not just call this v3 and be explicit about the change? I'm not sure that we gain much by trying to sell it as a minor update. Based on the work that @zUniQueX has done, it sounds like there are at least a handful of Jetty 10 changes that we'd have to reason about. By upgrading to Jetty 10 + Java 11 as a major Dropwizard version, it seems to me that it better communicates the magnitude of the change.

[joschi]

From other PRs and issues I thought you would highly prefer semantic versioning and therefore wouldn't want to make such an upgrade in 2.2.x 😉

@zUniQueX And you'd be absolutely right about it. 😅

Thanks to yours and @pstackle's feedback I'm convinced that the jump to Jetty 10 and Java 11 makes more sense for Dropwizard 3.x and the migration to Jakarta EE in Dropwizard 4.x. 👍

[joschi]

FYI, I've enabled GitHub Discussions in this repository and converted this issue into a discussion.

[zUniQueX]

Unfortunately we cannot upgrade Jetty to 10.0.x due to a logback issue. A PR for this is already raised: qos-ch/logback#532

I filed a bug report about the request log behaviour change from Jetty 11.0.7 to 11.0.8 and after the discussion I wouldn't want to ship Dropwizard 3.x.x and 4.x.x without the linked PR merged at logback and backported to logback 1.3.x with the jakarta namespace.

The comments of the Jetty team also make me a bit uneasy about logback-access and I think we could discuss, if we should provide a request log implementation for Jetty's classes.

Jetty does processing of the request anyway, so why shouldn't Jetty also do the logging? 😉

[joschi]

@zUniQueX Did the release of Jetty 10.0.9 and 11.0.9 help moving this issue forward?

https://github.com/eclipse/jetty.project/releases/tag/jetty-10.0.9
https://github.com/eclipse/jetty.project/releases/tag/jetty-11.0.9

[zUniQueX]

@joschi This lets us upgrade Jetty to 11.0.9. As I stated in the renovate PR for Jetty 11.0.8, they changed their handling in the request log and broke support for logback's RequestLogImpl. This is fixed in 11.0.9. However, the issue preventing us upgrading to Jetty 10.x is at logback's side, so this release wouldn't help us there.

Currently, upgrading to Jetty 10.x with logback seems only possible with a workaround. My PR on this provides a seperate commit for the logback related changes.

[brentryan]

Why all the concerns around javax vs jakarta? We've been running dropwizard 2.x for a while now and we enforce using only jakarta dependencies for everything. Maybe I'm missing something.

                                <bannedDependencies>
                                    <excludes>
                                        <!-- This should not exist as it will force SLF4J calls to be delegated to log4j -->
                                        <exclude>org.slf4j:slf4j-log4j12</exclude>
                                        <!-- This should not exist as it will force SLF4J calls to be delegated to jul -->
                                        <exclude>org.slf4j:slf4j-jdk14</exclude>
                                        <!-- Ensure only the slf4j binding for logback is on the classpath -->
                                        <exclude>log4j:log4j</exclude>
                                        <!-- As recommended from the slf4j guide, exclude commons-logging -->
                                        <exclude>commons-logging:commons-logging</exclude>
                                        <!-- Replaced with jakarta.activation:jakarta.activation-api -->
                                        <exclude>javax.activation:javax.activation-api</exclude>
                                        <!-- Replaced with jakarta.servlet:jakarta.servlet-api -->
                                        <exclude>javax.servlet:javax.servlet-api</exclude>
                                        <!-- Replaced with jakarta.validation:jakarta.validation-api -->
                                        <exclude>javax.validation:validation-api</exclude>
                                        <!-- Replaced with jakarta.xml.bind:jakarta.xml.bind-api -->
                                        <exclude>javax.xml.bind:jaxb-api</exclude>
                                        <!-- Replaced with jakarta.ws.rs:jakarta.ws.rs-api -->
                                        <exclude>javax.ws.rs:javax.ws.rs-api</exclude>
                                    </excludes>
                                </bannedDependencies>

[zUniQueX]

Dropwizard 2.x already uses for example jakarta.ws.rs-api, but the classes of the currently used version reside in the javax namespace, e.g. javax.ws.rs.GET. When updating the dependency to the next major version, the namespace changes, such that the class' FQDN is now jakarta.ws.rs.GET. To update the library, we therefore have to change all imports in our classes using this dependency.

[zUniQueX]

@dropwizard/committers Does anyone want to integrate further breaking changes into Dropwizard 3.0.0? If not, I think we're good to go, since we could now introduce the Automatic-Module-Names.

[mattnelson]

There were some breaking changes proposed here that could be 3.x candidates.

#4211 (comment)

[zUniQueX]

@mattnelson That's correct, haven't thought about that. As you suggested, we could introduce new conversion words in 3.0.x. Would you mind providing a follow-up PR for that?

[pgr0ss]

Do you have an idea of when v3/v4 will be released? We've noticed a lot of other (non-dropwizard) projects are now pulling in slf4j v2 in their upgrades (sometimes as part of minor or tiny releases) and it seems like we need v3/v4 of dropwizard for it to work properly. Thanks!

[zUniQueX]

Hi @pgr0ss. We don't have an estimated date, when v3 and v4 will be released.

We just disabled JMX to give our users the ability to override the SLF4J and Logback dependencies.