Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stick TASTy reading behind a -Ytasty-reader flag #9293

Merged
merged 1 commit into from Nov 2, 2020
Merged

Stick TASTy reading behind a -Ytasty-reader flag #9293

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/internal/tastyreader.md
View file
Open in desktop
Expand Up @@ -8,6 +8,8 @@ TASTy is an intermediate representation of a Scala program after type checking a

### Compiler flags

- `-Ytasty-reader` enables the support for reading Scala 3's TASTy files.

- `-Ydebug-tasty` enables rich output when traversing tasty files, important for tracing the history of events when diagnosing errors.

- `-Ytasty-no-annotations` ignores all annotations on tasty symbols, may be useful for ignoring complex annotations that are unsupported, but will prevent safety checks for pattern matching.
Expand Down
1 change: 1 addition & 0 deletions src/compiler/scala/tools/nsc/settings/ScalaSettings.scala
View file
Open in desktop
Expand Up @@ -261,6 +261,7 @@ trait ScalaSettings extends StandardScalaSettings with Warnings { _: MutableSett
val YmacroFresh = BooleanSetting ("-Ymacro-global-fresh-names", "Should fresh names in macros be unique across all compilation units")
val YmacroAnnotations = BooleanSetting ("-Ymacro-annotations", "Enable support for macro annotations, formerly in macro paradise.")
val YtastyNoAnnotations = BooleanSetting("-Ytasty-no-annotations", "Disable support for reading annotations from TASTy, this will prevent safety features such as pattern match exhaustivity and reachability analysis.")
val YtastyReader = BooleanSetting("-Ytasty-reader", "Enable support for reading Scala 3's TASTy files, allowing consumption of libraries compiled with Scala 3 (provided they don't use any Scala 3 only features).")
val Yreplclassbased = BooleanSetting ("-Yrepl-class-based", "Use classes to wrap REPL snippets instead of objects", default = true)
val YreplMagicImport = BooleanSetting ("-Yrepl-use-magic-imports", "In the code that wraps REPL snippets, use magic imports rather than nesting wrapper object/classes", default = true)
val Yreploutdir = StringSetting ("-Yrepl-outdir", "path", "Write repl-generated classfiles to given output directory (use \"\" to generate a temporary dir)" , "")
Expand Down
4 changes: 4 additions & 0 deletions src/compiler/scala/tools/nsc/symtab/classfile/ClassfileParser.scala
View file
Open in desktop
Expand Up @@ -81,6 +81,7 @@ abstract class ClassfileParser(reader: ReusableInstance[ReusableDataReader]) {
protected def moduleClass: Symbol = staticModule.moduleClass
protected val TASTYUUIDLength: Int = 16
private var sawPrivateConstructor = false
private var YtastyReader = false

private def ownerForFlags(jflags: JavaAccFlags) = if (jflags.isStatic) moduleClass else clazz

Expand Down Expand Up @@ -160,6 +161,7 @@ abstract class ClassfileParser(reader: ReusableInstance[ReusableDataReader]) {
this.clazz = clazz
this.staticModule = module
this.isScala = false
this.YtastyReader = settings.YtastyReader

val magic = in.getInt(in.bp)
if (magic != JAVA_MAGIC && file.name.endsWith(".sig")) {
Expand Down Expand Up @@ -1114,6 +1116,8 @@ abstract class ClassfileParser(reader: ReusableInstance[ReusableDataReader]) {
case tpnme.ScalaATTR =>
isScalaRaw = true
i = numAttrs
case tpnme.TASTYATTR if !YtastyReader =>
MissingRequirementError.signal(s"Add -Ytasty-reader to scalac options to parse the TASTy in $file")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dwijnand Why is it an error to unpickle a class file that has TASTY while running without -Ytasty-reader? How about silently skipping the attribute in this case?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, the idea was to fail eagerly when people unknowingly end up with Scala 3 classfiles on their classpath.

Skipping the attribute would mean the compiler treats Scala 3 classfiles like Java classfiles, I cannot imagine this would lead to a usable behavior?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks! So, there cannot be a scenario where there's regular Scala 2 metadata and TASTY?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, how about the case where a Scala 3 classfile is on the class path but not referenced directly? It could be forced because you're referencing a Scala 2 symbol directly, which then depends on a symbol from a Scala 3 jar. In that case, you should be able to safely consume the Scala 2 classes without adding -Ytasty-reader, I think. (We're running into this error in the wild, but I have not yet figured out how exactly.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there cannot be a scenario where there's regular Scala 2 metadata and TASTY?

That's right, Scala 3 doesn't emit ScalaSignatures (Scala 2 pickles), Scala 2 doesn't emit TSATy.

Scala 2 symbol which depends on a symbol from a Scala 3 jar

That's a good point. I haven't tested it, but it seems anything that depends on a Scala 3 jar "pollutes" the entire classpath.

Generally, publishing a Scala 2.13 library that depends on a Scala 3 library is discouraged, see https://docs.scala-lang.org/scala3/guides/migration/compatibility-classpath.html#disclaimer-for-library-maintainers. But in controlled environments this could be OK.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(The dependency in our case was pulled in by ammonite -- see cs resolve -t com.lihaoyi:ammonite-compiler-interface_2.13.6:2.4.0.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @lihaoyi

└─ com.lihaoyi:ammonite-compiler-interface_2.13.6:2.4.0
   └─ com.lihaoyi:ammonite-cross-23-util_2.13:2.4.0
      ├─ com.lihaoyi:ammonite-cross-23-ops_2.13:2.4.0
      │  ├─ com.lihaoyi:os-lib_2.13:0.7.2
      │  │  └─ com.lihaoyi:geny_2.13:0.6.5
      │  └─ org.scala-lang.modules:scala-collection-compat_2.13:2.4.1
      │     └─ org.scala-lang:scala-library:2.13.4 -> 2.13.5
      ├─ com.lihaoyi:fansi_3:0.2.14
      │  └─ com.lihaoyi:sourcecode_3:0.2.7
      ├─ com.lihaoyi:pprint_3:0.6.6
      │  ├─ com.lihaoyi:fansi_3:0.2.14
      │  │  └─ com.lihaoyi:sourcecode_3:0.2.7
      │  └─ com.lihaoyi:sourcecode_3:0.2.7
      └─ org.scala-lang:scala3-library_3:3.0.0
         └─ org.scala-lang:scala-library:2.13.5

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexarchambault is probably the expert here. I think we recently resolved an issue around accidental Scala3 dependencies in Ammonite? com-lihaoyi/Ammonite#1215

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks like that should fix it! Would be great to get a release out with this fix. I've seen several places mention this crash while I was trying to figure out what was going on. (Sounds like the canonical issue is com-lihaoyi/Ammonite#1214)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we probably should. @alexarchambault would you be able to tag a 2.4.1 release?

case tpnme.TASTYATTR =>
isTASTY = true
TASTYAttrLen = attrLen
Expand Down
4 changes: 2 additions & 2 deletions src/tastytest/scala/tools/tastytest/TastyTest.scala
View file
Open in desktop
Expand Up @@ -60,7 +60,7 @@ object TastyTest {

private def scalacPos(out: String, sourceRoot: String, additionalSettings: Seq[String], sources: String*): Try[Unit] = {
println(s"compiling sources in ${yellow(sourceRoot)} with scalac.")
successWhen(Scalac.scalac(out, additionalSettings, sources:_*))("scalac failed to compile sources.")
successWhen(Scalac.scalac(out, "-Ytasty-reader" +: additionalSettings, sources:_*))("scalac failed to compile sources.")
}

private def scalacNeg(out: String, additionalSettings: Seq[String], files: String*): Try[Unit] = {
Expand Down Expand Up @@ -89,7 +89,7 @@ object TastyTest {
}
val compiled = Console.withErr(byteArrayStream) {
Console.withOut(byteArrayStream) {
Scalac.scalac(out, additionalSettings, source)
Scalac.scalac(out, "-Ytasty-reader" +: additionalSettings, source)
}
}
byteArrayStream.flush()
Expand Down