ZipArchiveTest and ZipAndJarFileLookupFactoryTest fail under _some_ circumstances

[ansvonwa]

Reproduction steps

Scala version: 2.13.10, also latest 2.13.x -> 8fa1e7c1
Sbt version: 1.7.1
Java 8 and 11
Maybe you need java-atk-wrapper.jar or similar on your classpath, see below.

sbt test

or

sbt testOnly scala.reflect.io.ZipArchiveTest scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest

or

sbt junit / Test / testOnly scala.reflect.io.ZipArchiveTest scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest

or
in Intellij: open respective file, and run via Debug. (not Run!)

Output:

sbt:root> testOnly scala.reflect.io.ZipArchiveTest scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest
...
[info] Test run scala.reflect.io.ZipArchiveTest started
...
[info] Test scala.reflect.io.ZipArchiveTest.manifest resources just works started
[error] Test scala.reflect.io.ZipArchiveTest.manifest resources just works failed: java.lang.AssertionError: null, took 0.028 sec
[error]     at scala.reflect.io.ZipArchiveTest.$anonfun$manifest resources just works$2(ZipArchiveTest.scala:60)
[error]     at scala.reflect.io.ZipArchiveTest.$anonfun$manifest resources just works$2$adapted(ZipArchiveTest.scala:58)
[error]     at scala.util.Using$.$anonfun$resources$2(Using.scala:298)
[error]     at scala.util.Using$.resource(Using.scala:261)
[error]     at scala.util.Using$.$anonfun$resources$1(Using.scala:297)
[error]     at scala.util.Using$.resource(Using.scala:261)
[error]     at scala.util.Using$.resources(Using.scala:296)
[error]     at scala.reflect.io.ZipArchiveTest.manifest resources just works(ZipArchiveTest.scala:58)
[error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error]     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]     at java.lang.reflect.Method.invoke(Method.java:566)
[error]     ...
[info] Test run scala.reflect.io.ZipArchiveTest finished: 1 failed, 0 ignored, 5 total, 0.093s
[info] Test run scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest started
[info] Test scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest.cacheInvalidation started
[info] Test scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest.manifest classpath entry works started
[error] Test scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest.manifest classpath entry works failed: java.lang.AssertionError: null, took 0.024 sec
[error]     at scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest.$anonfun$manifest classpath entry works$5(ZipAndJarFileLookupFactoryTest.scala:91)
[error]     at scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest.$anonfun$manifest classpath entry works$5$adapted(ZipAndJarFileLookupFactoryTest.scala:88)
[error]     at scala.util.Using$.$anonfun$resources$5(Using.scala:328)
[error]     at scala.util.Using$.resource(Using.scala:261)
[error]     at scala.util.Using$.$anonfun$resources$4(Using.scala:327)
[error]     at scala.util.Using$.resource(Using.scala:261)
[error]     at scala.util.Using$.$anonfun$resources$3(Using.scala:326)
[error]     at scala.util.Using$.resource(Using.scala:261)
[error]     at scala.util.Using$.resources(Using.scala:325)
[error]     at scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest.manifest classpath entry works(ZipAndJarFileLookupFactoryTest.scala:88)
[error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error]     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]     at java.lang.reflect.Method.invoke(Method.java:566)
[error]     ...
[info] Test run scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest finished: 1 failed, 0 ignored, 2 total, 0.406s
[error] Failed: Total 8, Failed 2, Errors 0, Passed 6
[error] Failed tests:
[error] 	scala.tools.nsc.classpath.ZipAndJarFileLookupFactoryTest
[error] 	scala.reflect.io.ZipArchiveTest
[error] (junit / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful

Problem

The problem does not occur in some other circumstances (notably the CI). Also @som-snytt wasn't able to reproduce it immediately.

I don't know exactly, what the bug is or how to trigger/prevent it, but I found that the manifestAt returns the wrong manifest.
Adding the following print statement yields interesting insight.

diff --git a/test/junit/scala/reflect/io/ZipArchiveTest.scala b/test/junit/scala/reflect/io/ZipArchiveTest.scala
index ec7ede4348..fd49d6ab20 100644
--- a/test/junit/scala/reflect/io/ZipArchiveTest.scala
+++ b/test/junit/scala/reflect/io/ZipArchiveTest.scala
@@ -50,7 +50,11 @@ class ZipArchiveTest {
     assertThrown[IOException](_.getMessage.contains(f.toString))(fza.iterator)
   }
 
-  private def manifestAt(location: URI): URL = ScalaClassLoader.fromURLs(List(location.toURL), null).getResource("META-INF/MANIFEST.MF");
+  private def manifestAt(location: URI): URL = {
+    val url = ScalaClassLoader.fromURLs(List(location.toURL), null).getResource("META-INF/MANIFEST.MF")
+    println(url)
+    url
+  }
 
   // ZipArchive.fromManifestURL(URL)
   @Test def `manifest resources just works`(): Unit = {

This prints
jar:file:/home/anselm/Downloads/idea-IC-221.4501.155/plugins/java/lib/rt/debugger-agent.jar!/META-INF/MANIFEST.MF
in Intellij's debugger and
jar:file:/usr/share/java/java-atk-wrapper.jar!/META-INF/MANIFEST.MF
in my sbt.

So, clearly, the ScalaClassLoader.getResource returns a manifest of a different jar than set by fromURLs.
This may be a bug in ScalaClassLoader or the classloader may be used in a wrong way in manifestAt().

I hope somebody is able to reproduce this behaviour with the given hints. (Maybe add another jar with another MANIFEST to classpath or so?)

[liang3zy22]

I have same issue in my environment. The two cases failed every time. My environment is as below:
scala 2.13.10 and 2.13.x.
sbt 1.7.2
java 11.0.16

I thought it was my openjdk configuration problem. I will try to get more info.

[SethTisue]

relevant: https://stackoverflow.com/questions/58236241/openjdk-11-0-4-has-java-atk-wrapper-jar-in-the-classpath

on my system,

scala 2.13.10> getClass.getResource("META-INF/MANIFEST.MF")
val res0: java.net.URL = null

I guess it doesn't return null on affected systems?

if "Stephen C" is correct then:

the problem would appear to be a bug in the OpenJDK packages for Debian / Ubuntu

regardless, we'd happily accept a patch that makes the test more robust