Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Signed Jar verification fails from a nested Jar under Oracle Java 17 #28837

Closed
Tracked by #36135
thelateperseus opened this issue Nov 29, 2021 · 54 comments
Closed
Tracked by #36135
Assignees
Labels
type: bug A general bug
Milestone

Comments

@thelateperseus
Copy link

When running a Spring Boot app as a fat Jar under Java 17, using the Bouncy Castle provider results in an exception SecurityException: JCE cannot authenticate the provider BC with cause IllegalStateException: zip file closed. Any use of the provider seems to trigger the exception, e.g.

Cipher.getInstance("AES/CBC/PKCS5Padding","BC");

I have created a sample Spring Boot app that reproduces the problem.

I stepped through the code and I believe the problem is caused by the Spring Boot JarURLConnection returning an already closed Jar file from getJarFile(). I think this relates to issues #17127 and #25538, but I could be wrong.

This same issue does not occur under Java 11, so I assume something has changed in JarVerifier.verifySingleJar between Java 11 and 17.

The exception stack trace is:

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: java.lang.SecurityException: JCE cannot authenticate the provider BC
        at java.base/javax.crypto.Cipher.getInstance(Cipher.java:722)
        at java.base/javax.crypto.Cipher.getInstance(Cipher.java:642)
        at com.example.bctest.BctestApplication.main(BctestApplication.java:14)
        ... 8 more
Caused by: java.lang.IllegalStateException: zip file closed
        at java.base/java.util.zip.ZipFile.ensureOpen(ZipFile.java:831)
        at java.base/java.util.zip.ZipFile.getManifestName(ZipFile.java:1057)
        at java.base/java.util.zip.ZipFile$1.getManifestName(ZipFile.java:1100)
        at java.base/javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:461)
        at java.base/javax.crypto.JarVerifier.verifyJars(JarVerifier.java:317)
        at java.base/javax.crypto.JarVerifier.verify(JarVerifier.java:260)
        at java.base/javax.crypto.ProviderVerifier.verify(ProviderVerifier.java:130)
        at java.base/javax.crypto.JceSecurity.verifyProvider(JceSecurity.java:190)
        at java.base/javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:218)
        at java.base/javax.crypto.Cipher.getInstance(Cipher.java:718)
        ... 10 more
@snicoll
Copy link
Member

snicoll commented Nov 29, 2021

See #28157.

@mikegike
Copy link

#28150 The ticket doesn’t consider the used Java Version.
When using OpenJDK, what often Linux does, there is no Signature check for used provider jars.
Oracle JDK checks the Signature for used providers.

See: https://www.baeldung.com/oracle-jdk-vs-openjdk
Oracle has always required third party cryptographic providers to be signed by a known certificate, while cryptography framework in OpenJDK has an open cryptographic interface, which means there is no restriction as to which providers can be used

I think that is why there was a problem to recreate the issue.

@thelateperseus
Copy link
Author

thelateperseus commented Nov 29, 2021

I can reproduce the issue with my test app (Spring Boot 2.5.7) on all the following configurations. Note that these are all different physical computers - WSL is running on a different Windows 10 machine.

  • Oracle JDK 17.0.1 on Windows 10
  • Oracle JDK 17.0.1 on Unubtu 20.04 (Windows Subsystem for Linux on Windows 10)
  • Oracle JDK 17.0.1 on Windows Server 2016

However, the problem does not occur using OpenJDK 17.0.0 on Ubuntu 20.04 (Windows Subsystem for Linux).

Issue #28157 has a similar exception message, but a different root cause exception. I can reproduce my issue across multiple machines and operating systems, and the root cause is always "zip file closed." Each of these machines has downloaded a separate copy of the bouncy castle jar, so I don't believe it's corrupt.

@thelateperseus thelateperseus changed the title Bouncy Castle Jar verification fails from a fat Jar under Java 17 Bouncy Castle Jar verification fails from a fat Jar under Oracle Java 17 Nov 29, 2021
@wilkinsona
Copy link
Member

Thanks for the sample, @thelateperseus. I've reproduced the problem on macOS using Oracle JDK 17.0.1:

$ java -version
java version "17.0.1" 2021-10-19 LTS
Java(TM) SE Runtime Environment (build 17.0.1+12-LTS-39)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.1+12-LTS-39, mixed mode, sharing)
$ java -jar build/libs/spring-boot-bouncy-castle-0.0.1-SNAPSHOT.jar 
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: java.lang.SecurityException: JCE cannot authenticate the provider BC
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:722)
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:642)
	at com.example.bctest.BctestApplication.main(BctestApplication.java:14)
	... 8 more
Caused by: java.lang.IllegalStateException: zip file closed
	at java.base/java.util.zip.ZipFile.ensureOpen(ZipFile.java:831)
	at java.base/java.util.zip.ZipFile.getManifestName(ZipFile.java:1057)
	at java.base/java.util.zip.ZipFile$1.getManifestName(ZipFile.java:1100)
	at java.base/javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:461)
	at java.base/javax.crypto.JarVerifier.verifyJars(JarVerifier.java:317)
	at java.base/javax.crypto.JarVerifier.verify(JarVerifier.java:260)
	at java.base/javax.crypto.ProviderVerifier.verify(ProviderVerifier.java:130)
	at java.base/javax.crypto.JceSecurity.verifyProvider(JceSecurity.java:190)
	at java.base/javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:218)
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:718)
	... 10 more

@wilkinsona wilkinsona added this to the 2.5.x milestone Nov 29, 2021
@wilkinsona wilkinsona added type: bug A general bug and removed status: waiting-for-triage An issue we've not yet triaged labels Nov 29, 2021
@philwebb
Copy link
Member

😭

@philwebb
Copy link
Member

I've managed to find some time to dig into this today and unfortunately we have more than just the "zip file closed" issue to solve. I patched a local build so that close() is no longer called early and we get a different exception.

Exception in thread "main" java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: java.lang.SecurityException: JCE cannot authenticate the provider BC
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:722)
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:642)
	at com.example.bctest.BctestApplication.main(BctestApplication.java:14)
	... 8 more
Caused by: java.util.jar.JarException: The JCE Provider jar:file:/Volumes/Data/projects/spring-boot/samples/spring-boot-bouncy-castle/build/libs/spring-boot-bouncy-castle-0.0.1-SNAPSHOT.jar!/BOOT-INF/lib/bcprov-jdk15on-1.69.jar!/ is not signed.
	at java.base/javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:464)
	at java.base/javax.crypto.JarVerifier.verifyJars(JarVerifier.java:317)
	at java.base/javax.crypto.JarVerifier.verify(JarVerifier.java:260)
	at java.base/javax.crypto.ProviderVerifier.verify(ProviderVerifier.java:130)
	at java.base/javax.crypto.JceSecurity.verifyProvider(JceSecurity.java:190)
	at java.base/javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:218)
	at java.base/javax.crypto.Cipher.getInstance(Cipher.java:718)
	... 10 more

It took a bit more digging to get to the bottom of it (not helped by the lack of source code for JarVerifier). I think what happens is verifySingleJar is trying to check if the nested bcprov-jdk15on-1.69.jar jar is signed. To do that, it has the following code (more or less):

if (!jarManifestNameChecked && SharedSecrets.getJavaUtilZipFileAccess().getManifestName(jf, true) == null) {
    throw new JarException("The JCE Provider " + jarURL.toString() + " is not signed.");
}

The SharedSecrets.getJavaUtilZipFileAccess().getManifestName(jf, true) call ends up calling ZipFile.getManifestName(onlyIfSignatureRelatedFiles). Unfortunately this is a private method so our JarFile implementation can't override it. This means that rather than checking the nested jar, we end up checking the root jar. Since there are not signature files in the root jar the getManifestName method returns null.

I've managed to hack around the problem by adding an empty META-INF/BOOT.SF file in the fat jar. This is enough to get getManifestName to return a non null value and things then start up fine.

I'm not sure that this is really a good long term suggestions. I'll flag the issue for team attention to see if anyone has any bright ideas. Ideally we'd like getManifestName to be a protected method, then we could override it.

@philwebb philwebb added the for: team-attention An issue we'd like other members of the team to review label Dec 18, 2021
@philwebb
Copy link
Member

Hacked up code is at https://github.com/philwebb/spring-boot/tree/gh-28837

@zezulka000
Copy link

I am having same issue on spring boot 2.6.2 and oracle java 17 (oracle java 11 is working, same as open JDK versions as listed above).
I excluded dependency to bouncy castle jar, and added external classpath to fat jar, but problem remained the same:
java.lang.SecurityException: JCE cannot authenticate the provider BC
Caused by: java.lang.IllegalStateException: zip file closed

@zeroleak
Copy link

same blocking issue here

@evgenyigumnov
Copy link

I have the same blocking issue :(

@snicoll
Copy link
Member

snicoll commented Jan 18, 2022

@zeroleak @evgenyigumnov sorry about that but please use the reaction on the original description rather than this.

@evgenyigumnov
Copy link

@zeroleak @evgenyigumnov sorry about that but please use the reaction on the original description rather than this.

@snicoll what do you mean? I can not use Oracle JDK 11 version. I have to use 17 version.

@snicoll
Copy link
Member

snicoll commented Jan 18, 2022

I mean that "I have the same issue" is not very helpful and send a notification to the 3K watchers of this repository. If you want to indicate you're affected by this issue without any extra information, please rather use the reaction (👍) in the original description above.

@varkychen
Copy link

varkychen commented Feb 3, 2022

I could work around this issue by running my Spring Boot app using an exploded directory format. This is the recommended approach by Spring Boot for container images → Container Images

@wilkinsona wilkinsona added the for: team-meeting An issue we'd like to discuss as a team to make progress label May 13, 2022
@wilkinsona wilkinsona modified the milestones: 2.5.x, 2.6.x May 19, 2022
@wilkinsona wilkinsona removed the for: team-meeting An issue we'd like to discuss as a team to make progress label May 25, 2022
@philwebb
Copy link
Member

For anyone watching this issue I just pushed a fix for #29356 which should allow <requiresUnpack> to work.

@thelateperseus
Copy link
Author

I've tested this with my sample Spring Boot app and it works now. I had to change the Spring Boot version in build.gradle to 2.6.9 and add this section to the end of the file:

bootJar {
    requiresUnpack '**/bcprov-jdk15on-*.jar'
}

@jarekczek
Copy link

A comment appeared under bug entry at Oracle. They conclude it requires investigation on Spring Boot side:

I think this issue would have to be evaluated in the spring-boot project. It appears that the spring-boot project extends the java.util.jar.JarFile. That extended JarFile instances get used by the spring-boot launcher code. There's a specific piece of code in the extended class implementation here which immediately calls close() on the java.util.jar.JarFile https://github.com/spring-projects/spring-boot/blob/main/spring-boot-project/spring-boot-tools/spring-boot-loader/src/main/java/org/springframework/boot/loader/jar/JarFile.java#L131 which thus marks the JarFile instance as closed. I am not familiar with spring-boot and why it does this.

@ab61636452

This comment was marked as duplicate.

@linnex81
Copy link

linnex81 commented Sep 22, 2023

The only workaround I could find so far for this issue is the following (quite hacky). You need to run the verification of BouncyCastle before the Spring Boot "magic" starts as @jarekczek suggested - while still keeping the fat-jar (mostly).

Please note: in Oracle JDK 17 the verification result is stored in a permanent HashMap. If they change it to WeakHashMap somehow this will not work anymore!

You can do so as follows:

Create your own launcher derived from JarLauncher

public class MyLauncher extends JarLauncher {    
  public static void main(String[] args) throws Exception {    
// perform the registering / verification (cached) of BouncyCastle prior to any SpringMagic    
    Security.addProvider(new BouncyCastleProvider());    
    Cipher c = Cipher.getInstance("AES/CBC/PKCS5Padding","BC");    
    new MyLauncher().launch(args);    
  }    
}

exclude bcprov from your fat jar (you have to -cp it separately along with your fat jar for launching - but at least it's the only other jar)

<exclude>
  <groupId>org.bouncycastle</groupId>
  <artifactId>bcprov-jdk15on</artifactId>
</exclude>

use e.g. maven-antrun-plugin to repack your fat-jar to include your "MyLauncher" in the right place of the jar

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-antrun-plugin</artifactId>
  <version>3.1.0</version>
  <executions>
    <execution>
      <id>copy-and-repair-zips</id>
      <phase>package</phase>
      <goals>
        <goal>run</goal>
      </goals>
      <configuration>
        <target>
          <zip destfile="${project.build.directory}/${project.artifactId}-${project.version}_mod.jar" keepcompression="true">
            <zipfileset src="${project.build.directory}/${project.artifactId}-${project.version}.jar"/>
            <zipfileset dir="${project.build.outputDirectory}/launcher" includes="MyLauncher.class" fullpath="launcher/MyLauncher.class" />
          </zip>
          <move file="${project.build.directory}/${project.artifactId}-${project.version}_mod.jar" tofile="${project.build.directory}/${project.artifactId}-${project.version}.jar" overwrite="true" />
          <!-- now the modified jar is available  -->
        </target>
      </configuration>
    </execution>
  </executions>
</plugin>

Hopefully this gets fixed with 3.2.x or earlier - thanks!

(sorry, I could not get the code formatting to work)

@wilkinsona
Copy link
Member

You need to use triple backticks, as described in GitHub's documentation. I have edited your comment to do so.

@linnex81
Copy link

You need to use triple backticks, as described in GitHub's documentation. I have edited your comment to do so.

Thank you @wilkinsona. I only found the - supposedly outdated - hints to add two spaces at the end of the lines.

@philwebb philwebb changed the title Bouncy Castle Jar verification fails from a fat Jar under Oracle Java 17 Bouncy Castle Jar verification fails from a nested Jar under Oracle Java 17 Oct 3, 2023
@amalajone
Copy link

Hi @philwebb ..When it will be fixed ?..The work around given not working

philwebb added a commit that referenced this issue Oct 6, 2023
Update `spring-boot-loader-tests` with a test that checks verified
BouncyCastle jars can be loaded. Currently the Oracle JDK only supports
verification if the jar is unpacked.

See gh-28837
@philwebb
Copy link
Member

philwebb commented Oct 6, 2023

@amalajone We don't have an exact ETA for a fix, but we did push code for #37668 which was a substantial amount of work that will hopefully make it easier to fix this one. I've also just pushed some tests in 6c24ea0 which seem to suggest that the new nested jar support is supporting the <requiresUnpack> workaround.

@philwebb philwebb modified the milestones: 2.7.x, 3.2.x Oct 6, 2023
@philwebb philwebb changed the title Bouncy Castle Jar verification fails from a nested Jar under Oracle Java 17 Signed Jar verification fails from a nested Jar under Oracle Java 17 Oct 16, 2023
@philwebb philwebb self-assigned this Oct 16, 2023
@philwebb philwebb modified the milestones: 3.2.x, 3.2.0-RC1 Oct 16, 2023
@xudong-1990
Copy link

xudong-1990 commented Nov 15, 2023

I have the same problem.
My springboot version is 3.1.0
JDK: oracle jdk17.0.9

My solution is to replace Oracle JDK and use OpenJDK-based TencentKona instead. TencentKona does not verify the signature of the jar package, so the problem is solved (although this is not the best solution).

@terrancechen
Copy link

The bug report was accepted (The Oracle team did a fast and good job) and should be visible as JDK-8313742.

Oracle has fixed this issue on 11/06/2023 in version 17.0.11, please upgrade your JDK version.

@deep-upreti
Copy link

The bug report was accepted (The Oracle team did a fast and good job) and should be visible as JDK-8313742.

Oracle has fixed this issue on 11/06/2023 in version 17.0.11, please upgrade your JDK version.

Is this version released , I can see 17.0.9 as latest

@missence
Copy link

missence commented Jan 2, 2024

I am using the latest version of jdk17 and I still have the same error

@missence
Copy link

missence commented Jan 2, 2024

该错误报告已被接受(Oracle 团队做得又快又好),并且应该显示为JDK-8313742

Oracle已于2023年11月6日在17.0.11版本中修复了该问题,请升级您的JDK版本。

I am using the latest version of jdk17 and I still have the same error

@rosti-il
Copy link

Try JDK 17.0.10 released today. The JDK-8313742 is mentioned as fixed in the release notes of this JDK release:
https://www.oracle.com/java/technologies/javase/17-0-10-relnotes.html

@mars-men
Copy link

JDK 17.0.10 -> Effective for me.

@amalajone
Copy link

When we generate the package as jar and if we using spring 3.2.2 and Java 17.0.9 then issue resolved but same issue still there when we generate the package as war .

@amalajone
Copy link

When we generate the package as jar and if we using spring 3.2.2 and Java 17.0.9 then issue resolved but same issue still there when we generate the package as war .

@philwebb anyone reported same issue ?

@philwebb
Copy link
Member

philwebb commented Feb 2, 2024

@amalajone I don't think so. Please open a new issue with a reproducer and we can take a look.

@amalajone
Copy link

Ok sure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug A general bug
Projects
None yet
Development

No branches or pull requests