Cross publish native

[kpodsiad]

An attempt to cross publish munit-native & rest to Scala 3.

Needs Scala 3.1.1 to be released, otherwise macro mixing in MacroCompat2 won't work. It fails with the Experimental erased may only be used with a nightly or snapshot version of the compiler - see scala/scala3#13795 for more details.

[sideeffffect]

We can drop the -RC2 now 😃

[kpodsiad]

@sideeffffect @gabro I'll answer to #488 here. There's a problem with scalacheck - there is no scalacheck published for Scala 3 & Native 0.4.x, see maven. I tried various tricks with CrossVersion.for3Use2_13 and etc. but without any results.
When running ++3.1.1 testsNative/test I always ended with munit depending on 0.4_3 and munitScalacheck depending on the 0.4_2.13 which unfortunately doesn't work.

@sideeffffect if you want you can play with it to get ++3.1.1 testsNative/test passing. I'm not a multiversion multiplatform expert, maybe you can get it done ;)

There is a PR to support scala-native Scala 3 in scalacheck repo typelevel/scalacheck#868. Unfortunately, it's waiting for maintainers to look at it.

[kpodsiad]

I see 3 options:

• find a workaround for scalacheck scala3 & Native 0.4.x and made ++3.1.1 testsNative/test passing. Can give it another try but it won't be in this week :/
• wait for support scala-native Scala 3 build typelevel/scalacheck#868
• extract scalacheck related tests to separate module, then munit can be released without any hassle and munit-scalacheck will wait for support scala-native Scala 3 build typelevel/scalacheck#868

@gabro Are you in favor of any option?

[gabro]

Well if we have a viable workaround, sure, but it really depends on whether you or someone else can find it.

Reaching out to Typelevel about the status of Scalacheck and getting that PR merged upstream would be the preferable option for sure, while I'm not really keen on the idea of releasing MUnit without full Scalacheck support.

[sideeffffect]

I tried various tricks with CrossVersion.for3Use2_13 and etc. but without any results.
When running ++3.1.1 testsNative/test I always ended with munit depending on 0.4_3 and munitScalacheck depending on the 0.4_2.13 which unfortunately doesn't work.

I think I've also hit this issue

[error] Modules were resolved with conflicting cross-version suffixes in ProjectRef(uri("file:/home/ondra/Projects/sbt-locales/"), "apiNative"):
[error]    org.scala-native:windowslib_native0.4 _3, _2.13
...

Please read more here:
portable-scala/portable-scala-reflect#23 (comment)

[avdv]

Note, there is also a problem with scala-native itself you would need to workaround: scala-native/scala-native#2543

[sideeffffect]

For a workaround for the

[error] Modules were resolved with conflicting cross-version suffixes in ProjectRef(uri("file:/home/ondra/Projects/sbt-locales/"), "apiNative"):
[error]    org.scala-native:windowslib_native0.4 _3, _2.13

issue, see cquiroz/sbt-locales#190

[avdv]

Using an inline conditional to work around issue scala-native/scala-native#2543 would be possible:

diff --git a/munit/native/src/main/scala/munit/internal/PlatformCompat.scala b/munit/native/src/main/scala/munit/internal/PlatformCompat.scala
index c08636e..e625d88 100644
--- a/munit/native/src/main/scala/munit/internal/PlatformCompat.scala
+++ b/munit/native/src/main/scala/munit/internal/PlatformCompat.scala
@@ -38,7 +38,7 @@ object PlatformCompat {
 
   def isJVM: Boolean = false
   def isJS: Boolean = false
-  def isNative: Boolean = true
+  inline val isNative = true // Scala 3 syntax
 
   def newRunner(
       taskDef: TaskDef,
diff --git a/munit/shared/src/main/scala-3/munit/internal/Compat.scala b/munit/shared/src/main/scala-3/munit/internal/Compat.scala
index 7e489e2..92b507d 100644
--- a/munit/shared/src/main/scala-3/munit/internal/Compat.scala
+++ b/munit/shared/src/main/scala-3/munit/internal/Compat.scala
@@ -7,7 +7,10 @@ object Compat {
   val LazyList = scala.LazyList
   def productElementNames(p: Product): Iterator[String] =
     p.productElementNames
-  def collectionClassName(i: Iterable[_]): String = {
-    i.asInstanceOf[{ def collectionClassName: String }].collectionClassName
+  inline def collectionClassName(i: Iterable[_]): String = {
+    if PlatformCompat.isNative then
+        i.getClass.getName // or just "Iterable" ?
+    else
+        i.asInstanceOf[{ def collectionClassName: String }].collectionClassName
   }
 }

[kpodsiad]

I spoke with Wojciech and there should be the Scala Native release at the end of this week. That release will contain fixes for problems spotted here, so I'll wait for it.

[armanbilge]

That release will contain fixes for problems spotted here, so I'll wait for it.

Do you mean for CrossVersion.for3Use2_13? I must strongly recommend against using this technique.

When a widely-used library (such as scalacheck) is depended on with CrossVersion.for3Use2_13 by another widely-used library (such as munit) this creates a binary-breaking division in the ecosystem: even after scalacheck is properly published for Native/Scala 3, this artifact is incompatible with the Native/2.13 artifact that munit is using, and thus the ecosystem will be split on this dependency.

For an example of how this plays out, I strongly recommend you read this thread.

• What to do about the Scala 3-specific artifact? scala-js/scala-js-dom#451

tl;dr scala-js-dom and all its downstream users (very nearly the entire Scala.js ecosystem) had to do a breaking version jump.

Further reading:

• https://gist.github.com/djspiewak/b8f2b4547951442102488964bc351cf9#5-dont-rely-on-withdottycompat
  Note that withDottyCompat became for3Use2_13

[kpodsiad]

@armanbilge thanks for this information. Just to confirm, the preferable solution is waiting until scalacheck is published properly for scala-native/scala3? In every other approach, munit ends up depending on the _2.13 artifact.

[armanbilge]

Just to confirm, the preferable solution is waiting until scalacheck is published properly for scala-native/scala3?

Yes, exactly. Preferable, as in doesn't break the ecosystem 😉

[SethTisue]

I just merged some PRs in the ScalaCheck repo that may help get things moving again.

[kpodsiad]

@SethTisue this PR still needs typelevel/scalacheck#868 (or equivalent) to be merged & new version released

[armanbilge]

@kpodsiad how would munit feel about scalacheck bumping to 3.1 on all platforms? See discussion in typelevel/scalacheck#868 (comment).

[kpodsiad]

@armanbilge I'm afraid that I'm not the right person to ask.

@olafurpg @gabro could you share your opinion as creators and maintainers?

[gabro]

IIUC this would only affect MUnit + Scalacheck on Scala Native.

If that's the case then I think it's fine (I'm assuming we can keep using the current version of Scalacheck for jvm/js targets in order to preserve compatibility with 3.0)

The only drawback is that downstream libraries targeting e.g. jvm and native would need to do the same (either depend on different versions of MUnit or break compatibility with 3.0)

I'll let @olafurpg call the shots, but it wouldn't be a terrible thing in my opinion

[olafurpg]

No objections from me to drop 3.0 support

[SethTisue]

IIUC this would only affect MUnit + Scalacheck on Scala Native
I'm assuming we can keep using the current version of Scalacheck for jvm/js targets in order to preserve compatibility with 3.0

one would assume so (I did too), but the assumption was called into question by @armanbilge at typelevel/scalacheck#868 (comment) , where he wrote:

these sorts of hacks seem to cause problems see e.g. portable-scala/sbt-crossproject#130

how strong an objection that is, I'm not sure — if we run into trouble, would it be minor trouble we could work around, or really serious trouble?

anyway, I hope that clarifies why Arman chose this phrasing above:

how would munit feel about scalacheck bumping to 3.1 on all platforms?

note the "on all platforms".

[armanbilge]

No one's talked me down, so let's escalate 😉

• Onwards and upwards typelevel/scalacheck#879

[armanbilge]

Scalacheck has now published a snapshot with Scala 3 / native support! I expect a stable release is imminent.

https://s01.oss.sonatype.org/content/repositories/snapshots/org/scalacheck/scalacheck_native0.4_3/1.16-0ac8005-SNAPSHOT/

[armanbilge]

Your move! 😁

https://repo1.maven.org/maven2/org/scalacheck/scalacheck_native0.4_3/

[kpodsiad]

@armanbilge Could you help me with JS tests? They suddenly started failing after I bumped scala-js to the 1.9.0. Although there is a clue what to do, I don't why it's needed at all and that's why I would appreciate help from someone with any scala.js experience ;)

[sjrd]

You're running into the fixes with compatibility concerns about regular expressions in Scala.js 1.7.0. There are suggested strategies on how to avoid MULTILINE in the regular expressions doc page of Scala.js.

This is not my repo, but I would humbly suggest upgrading Scala.js in a separate PR to be able to address these changes independently of the larger changes in this PR.

[sjrd]

It seems that the problematic regular expressions is at

munit/tests/shared/src/test/scala/munit/BaseFrameworkSuite.scala

Lines 85 to 86 in 36aeb75

	val elapsedTimePattern =
	Pattern.compile(" \\d+\\.\\d+s ?", Pattern.MULTILINE)

This regexp doesn't even use ^ nor $, and is only used with replaceAll (so not any anchored situation). The MULTILINE flag has no effect in that case, so it can safely be removed without any other change.

[armanbilge]

@kpodsiad does Seb's suggestion fix it (thank you!)? Otherwise I'll try and looking into it a bit later, I would really like to unblock this :)

[kpodsiad]

@armanbilge I'll try to look at this tomorrow or, more probably, next week.

Because of with Scala.js 1.8+ in
Scalacheck 1.16.0 is binary compatible with the 1.15.x and 1.14.x series. It is published for Scala 2.12, 2.13, and 3.1+ with Scala.js 1.8+ and Scala Native 0.4. This release is the first to support Scala 3 on the Native platform.

it seems like I need to either:

• leave the old version of scalacheck for JS and use the newly published 1.16.0 for Native and JVM.
• just add the linkerConfig for tests (scalaJSLinkerConfig ~= { _.withESFeatures(_.withESVersion(ESVersion.ES2018)) }) because it's needed only in tests and it won't affect library users I think.

[armanbilge]

Hum, if I understand #477 (comment) correctly you just need to remove that MULTILINE and it should work.

Ok, I'll give it a try.

[armanbilge]

Woops, I also see that this PR is targeting main instead of a series/0.7 branch. If these changes are released as a 1.0 milestone, then most of the ecosystem won't be able to upgrade.

[armanbilge]

Ok, put up a PR at:

• Publish for Scala 3 + Native #511

[armanbilge]

Hum, the multiline flag is needed. The suggestion to bump the ES version for tests-only would work without affecting downstreams, but it would prevent your tests from being able to catch uses of ES2018 features in the published code. Which is not so great.

scalaJSLinkerConfig ~= { _.withESFeatures(_.withESVersion(ESVersion.ES2018)) }

Update: my bad, the regex was different in 0.7.29 which is why MULTILINE was required. Seb is right, as always 😉

[kpodsiad]

@olafurpg approval will be gladly welcomed to fulfill formalities ;)

[armanbilge]

Woops, fyi we've got conflicts now.

[valencik]

Sorry, merge conflicts are my fault. I'll resolve those and then merge once CI passes.

[valencik]

After some discussion in Discord yesterday we're going to go ahead with this targeting 1.0 and soon cut a stable v1.0

These changes look reasonable to me, and CI is a nice shade of green :)