Nested jar URLs cannot be split and reassembled resulting in errors with projects that use this technique (such as JobRunr)

[rdehuyss]

Hi, I'm Ronald, the creator of JobRunr.

An issue was reported to us regarding the new nested jar support which broke JobRunr as it reads files from the classpath.

Inside an Uber jar, it is currently impossible to list the files within a directory using a NIO FileSystemProvider

I would expect the following to work:

var url = ((JarURLConnection)H2StorageProvider.class.getResource("migrations").openConnection()).getJarFileURL();
FileSystem fileSystem = FileSystems.newFileSystem(url.toURI(), Collections.emptyMap());
Path path = fileSystem.getPath("BOOT-INF/lib/jobrunr-6.3.3.jar!/org/jobrunr/storage/sql/common/migrations/");
boolean exists = Files.exists(path);
List<Path> list = Files.list(path).collect(Collectors.toList());
return "Root exists: " + exists + "; contents: " + list;

The FileSystem returned is of type class org.springframework.boot.loader.nio.file.NestedFileSystem.

For a reproducible scenario, please see https://github.com/rdehuyss/spring-boot-3.2-nested-jar-issue

[philwebb]

Thanks very much for the detailed report @rdehuyss. I think we've actually got two distinct bugs here so I've opened #38595 for the second.

I think I have a fix for the sample project provided in jobrunr/jobrunr#884 but I'm not sure about the reproducer you attached to this issue.

As it stands, that your sample is calling H2StorageProvider.class.getResource("migrations") which returns a URL in the form jar:nested:/the.jar/!BOOT-INF/lib/jobrunr-6.3.3.jar!/org/jobrunr/storage/sql/h2/migrations.

It then gets the JAR file URL which is nested:/the.jar/!BOOT-INF/lib/jobrunr-6.3.3.jar and creates a FileSystem from it. Finally, it calls getPath("BOOT-INF/lib/jobrunr-6.3.3.jar!/org/jobrunr/storage/sql/common/migrations/").

There are a couple of problems here:

1. The filesystem created is not a ZipFileSystem
2. The path include the nested jar in it.

If I change the sample as follows, things work.

@GetMapping("/get/from/nested/jar")
public String getFromNestedJar() throws IOException, URISyntaxException {
	loadFileSystemIfNecessary();
	Path path = fileSystem.getPath("org/jobrunr/storage/sql/common/migrations/");
	boolean exists = Files.exists(path);
	List<Path> list = Files.list(path).collect(Collectors.toList());
	return "Root exists: " + exists + "; contents: " + list;
}

private static void loadFileSystemIfNecessary() throws IOException, URISyntaxException {
	if (fileSystem == null) {
		URL resource = H2StorageProvider.class.getResource("migrations");
		var jarFileUrl = ((JarURLConnection) resource.openConnection()).getJarFileURL();
		fileSystem = FileSystems.newFileSystem(new URI("jar:" + jarFileUrl), Collections.emptyMap());
	}
}

I think this makes sense because if we remove nested JARs from the picture, the original sample wouldn't work. For example, the following code:

URL url = new URL("jar:file:/the.jar!/");
JarURLConnection connection = (JarURLConnection) url.openConnection();
URL jarFileUrl = connection.getJarFileURL();
FileSystem fileSystem = FileSystems.newFileSystem(jarFileUrl.toURI(), Collections.emptyMap());

Fails when FileSystems.newFileSystem is called with:

Exception in thread "main" java.lang.IllegalArgumentException: Path component should be '/'
	at java.base/sun.nio.fs.UnixFileSystemProvider.checkUri(UnixFileSystemProvider.java:81)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newFileSystem(UnixFileSystemProvider.java:90)
	at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:339)
	at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:288)
	at org.spring.boot.nested.Test.main(Test.java:16)

In other words, we always need the jar: scheme if we want to look at the jar contents.

I hope that makes sense and the fix works OK. If it doesn't, please let me know.

[rdehuyss]

Hi @philwebb - thanks for your swift intervention and input.

Just to be 100% sure, I changed the reproducer project to the above example you gave (still using Spring Boot 3.2.0). Is it possible I then encounter the issue about java.util.zip.ZipException: read CEN tables failed? I assume this will then be fixed in Spring Boot 3.2.1?

[philwebb]

That correct. There's a 3.2.1-SNAPSHOT available if you want to give that a try. It's in repo.spring.io/snapshot. You can use start.spring.io to generate a project with the maven config you need.

[rdehuyss]

Will do! Any idea on a release date already?

[bclozel]

@rdehuyss it's scheduled on December 21st, see https://github.com/spring-projects/spring-boot/milestones and https://calendar.spring.io

[rdehuyss]

I should have known that there is a release calendar 🤓.

[rdehuyss]

Hi @philwebb , @wilkinsona, @bclozel : I can confirm that JobRunr works again with Spring Boot 3.2.1-SNAPSHOT.

Has anybody already mentioned you guys are awesome?! Thanks for the swift handling and feedback. I ❤️ that I do not need to change anything in JobRunr 🎉 (which I was expecting).