Support for some Scala 3 syntactic constructs in Scala 2 with -Xsource:3

[smarter]

Not sure if this is the correct place to discuss this, but this is a heads-up that Scala 2.13.6 and 2.12.14 will support a subset of the Scala 3 syntax under -Xsource:3:

• Support ? as wildcard marker
• Support case in pattern bindings
• Support Scala 3 wildcard and renaming imports
• Allow soft keywords open and infix
• Support Scala 3 vararg splice syntax
• Support writing & instead of with in types

These are all just parser changes intended to ease cross-compilation and it would be great if metals/scalameta/scalafmt/scalafix/... could understand these constructs in a Scala 2 codebase without the user having to do anything in particular. Note that while all these changes are currently under -Xsource:3, some of them might end up being enabled by default in future Scala 2 releases to ease migration.

[tgodzik]

Thanks for the heads up! We have three choices here:

1. Add these features to Scala212 and Scala213 dialects, which would make it all easier to use and would not require any changes in other tools, but would mean that we would parse things with the assumption of -Xsource:3
2. Create separate dialects, which means we would have to detect -Xsource:3 it in every tool that uses scalameta parser
3. Modification of 1., where we would also create LegacyScala212 and LegacyScala213 dialect, which can be used explicitely whenever needed.

I would honestly opt for 3 as it would mean most users will not really be affected and would not need to do anything. Scalafmt, Scalafix, Mdoc do not need errors from the parser and we should be able to handle it Metals thanks to recent changes by @dos65

Opinions, anyone? @olafurpg @ckipp01 @kitbellew @mlachkar ?

[kitbellew]

@tgodzik scalafmt rewrite rules use these flags, so they had better be enabled in reality.

given the previous difficulty in supporting try (x + y).foo before 2.13.3, i'd say that scalameta needs to implement

case class ScalaVersion(major: Int, minor: Int, patch: Int)
object Dialect {
  ...
  def getByVersion(version: ScalaVersion): Dialect
}

maybe even add a sinceVersion: ScalaVersion (included) and untilVersion: Option[ScalaVersion] (excluded) to every flag explicitly (rather than in comments).

[dos65]

@tgodzik
It seems that the most problematic change here is Allow soft keywords open and infix.
Enabling them by default might break existing codebases implicitly and make some people upset if they upgrade scalafmt.

I think the second option will be the safest one. Anyway, people had to enable -Xsource:3 compiler option by themself so it's reasonable that they also will have to update .scalafmt.conf

[smarter]

Enabling them by default might break existing codebases implicitly and make some people upset if they upgrade scalafmt.
[...]
Anyway, people had to enable -Xsource:3 compiler option by themself so it's reasonable that they also will have to update .scalafmt.conf

I think the number of people upset by the latter will dwarf the number of people upset by the former :). Anyway, I'd like to note that this particular change is less important than the first two changes (? as wildcard and case in for comprehensions) which I'd really like to see widespread support for since we'd like to change the meaning of the old _ and case-less pattern bindings in Scala 3 (cf the respective PRs), which means first disallowing the old meaning, which means all cross-compiling codebase will need to switch to the new syntax.

[dos65]

@smarter

Oh, I mean a different thing. I'm only about that it would be better not to enable -Xsource:3 by default in Scala212 and Scala213 dialects.

Let's say someone uses 2.12.10 and decided to upgrade to the latest scalafmt/metals/scalafix version.
In case if it has infix in the codebase that will lead to the situation that everything compiles but tooling doesn't work correctly.

So I think the best option here would be the second one where each tool detects this flag/has a setting to enable new syntax.

[tgodzik]

@smarter

Oh, I mean a different thing. I'm only about that it would be better not to enable -Xsource:3 by default in Scala212 and Scala213 dialects.

Let's say someone uses 2.12.10 and decided to upgrade to the latest scalafmt/metals/scalafix version.
In case if it has infix in the codebase that will lead to the situation that everything compiles but tooling doesn't work correctly.

So I think the best option here would be the second one where each tool detects this flag/has a setting to enable new syntax.

Yeah, those soft keywords are not too safe and would lead to unexpected situations.

[smarter]

In case if it has infix in the codebase

You'd have to use it in a very specific position to run into an issue (on a line by itself just before a def), so it's likely to be extremely rare and the default won't be a big deal either way.

[kitbellew]

i'd vote for users explicitly telling the scalameta-integrating tool that they want 2.13.6' rather than plain vanilla 2.13.6. after all, after decades of using [_] and =>, people can survive without [?] or as.

[smarter]

Speaking of dialect, I assume that Scala3 will become the default dialect once Scala 3.0 is out? In that case whether or not a new dialect is used to support these constructs won't matter so much if the default already support them.

[kitbellew]

Speaking of dialect, I assume that Scala3 will become the default dialect once Scala 3.0 is out? In that case whether or not a new dialect is used to support these constructs won't matter so much if the default already support them.

I think scala3 will become the default dialect after Scala 3.0 adoption exceeds that of Scala 2.1x.

[tgodzik]

Speaking of dialect, I assume that Scala3 will become the default dialect once Scala 3.0 is out? In that case whether or not a new dialect is used to support these constructs won't matter so much if the default already support them.

I think scala3 will become the default dialect after Scala 3.0 adoption exceeds that of Scala 2.1x.

Also, we cannot possibly force everyone to set Scala 2 dialect in their projects. We will try to add some improvements so that users will know they need to add the new option.

[olafurpg]

I agree that the default dialect should remain Scala 2 for a while even after the 3.0 release. Changing the default dialect to Scala 3 could cause breakages for all Scala 2 users.

[tgodzik]

After a bit of thought. Can't we just use dialect for Scala3 in cases where user enables the new option? If it's supposed to make sure that the code will compile with Scala 3, it might be even better to use that dialect.

We wouldn't need to do anything in scalameta and in metals we can just check if the version is 2.13.6 or above and that the new setting is set. In that case we would parse everything automatically with the Scala 3 dialect.

[smarter]

Is the Scala 3 dialect a superset of the Scala 2 one? Otherwise this could be problematic since using -Xsource:3 does not mean that all your code can be parsed by Scala 3, you might have some Scala 2 macros in a scala-2/ folder for example.

[tgodzik]

Ach ok, i thought that the option is much more limiting. We'll need a new dialect then when 2.13.6 comes out

[smarter]

Well, looking at https://github.com/scalameta/scalameta/blob/main/scalameta/dialects/shared/src/main/scala/scala/meta/dialects/package.scala I think it might be OK since most of the things which are disallowed are already deprecated in Scala 2, however:

.withAllowXmlLiterals(false) // Scala 3: parser doesn't support xml

That's incorrect, we still support xml literals with scala-xml on the classpath like Scala 2.

.withAllowSymbolLiterals(false)

We still support them under a language flag, so I would leave that to true.