War deployment in standalone Tomcat causes memory leak (Metaspace)

[AlexieKA]

Steps to reproduce:

1. Create an empty application in https://start.spring.io/ with the following parameters:

• Spring Boot 2.5.4
• Packaging War
• Java 11
• Spring Web

2. Build the application (gradle build)
3. Deploy the resulting war in a standalone Tomcat
4. Perform several (5-10) restarts of the application with Tomcat manager
5. As a result, we observe a metaspace leak (if a value of XX:MaxMetaspaceSiz is set and was reached, we get an exception)

Analysis

1. Create Heap Dump (for example, using jvisualvm)
2. Open the dump file with jvisualvm or MemoryAnalyzer
3. Select all objects of ParallelWebappClassLoader type
4. The number of objects is equal to the number of restarts plus active application instances in Tomcat
5. Select «Path to GC Root» for inactive ParallelWebappClassLoader
6. We can see the Thread (reference to ParallelWebappClassLoader) is stored in ApplicationShutdownHooks. Because of this reference, the GC cannot delete this ParallelWebappClassLoader instance and free metaspace

Causes

1. The thread that holds ParallelWebappClassLoader is created in the constructor of the class SpringApplicationShutdownHook (line 74)
2. The object of SpringApplicationShutdownHook is created when loading the SpringApplication type (line 203)
3. Since every restart of the application (from the tomcat admin panel) create new classloader and initialize SpringApplication type, which causes memory leak.

[AlexieKA]

Current workaround (kotlin):

@WebListener
internal class CleanupServletContextListener : ServletContextListener {

    override fun contextInitialized(sce: ServletContextEvent) {
    }

    override fun contextDestroyed(sce: ServletContextEvent) {
        clearSpringApplicationShutdownHook()
    }

    private fun clearSpringApplicationShutdownHook() {
        val clazz = Class.forName("java.lang.ApplicationShutdownHooks")
        val field: Field = clazz.getDeclaredField("hooks")
        field.isAccessible = true
        @Suppress("UNUSED_VARIABLE", "UNCHECKED_CAST")
        val hooks: IdentityHashMap<Thread, Thread> = field.get(null) as IdentityHashMap<Thread, Thread>

        val ghosts =
            hooks.entries.asSequence().filter { it.key.name == "SpringApplicationShutdownHook" }.map { it.key }.toList()
        ghosts.forEach { hooks.remove(it) }

        field.isAccessible = false
    }
}

[bclozel]

Did you set up your WAR deployment as documented in the reference docs?

The shutdown hook should not be registered for WAR deployments.

If you believe there's an issue, can you provide a sample application (that we can clone and compile) that reproduces the issue?

Thanks

[wilkinsona]

I've reproduced this while looking at #27996. The problem is that loading SpringApplication always triggers the creation of a SpringApplicationShutdownHook which registers a shutdown hook. This hook is then never unregistered. We don't need to register it at all because, as @bclozel notes above, use of the shutdown hook is disabled by default in war deployments. This is a regression introduced by #26660, the fix for another regression

I think we should delay registration of the hook until the first application context is registered.