Parse annotation in jointly compiled Java source files and honour @Deprecated

[dwijnand]

Rebased (with a small merge conflict) @hrhino's work in #5892 & extended it to handle @Deprecated.

Fixes scala/bug#8928 ("Java parser skips over annotations"), though limited to no args annotations and annotations with literal arguments.
Fixes scala/bug#9617 ("Deprecations from Java may be not taken into account in mixed projects") which is the joint-compilation variant of scala/bug#10752 (which was just fixed).
Fixes scala/bug#8883 ("Treat java / scala deprecated annotations more uniformly") (IMO) with the deprecated attribute adding the java annotation & Symbol#isDeprecated considering both.

[hrhino]

Thanks for (hopefully) getting this over the finish line. Third time's the charm.

[lrytz]

Suggestions by @retronym how we could handle annotation arguments that scalac fails to interpret:

• have an opt-in warning
• drop the annotation
• replace the argument with a dummy value
• add a synthetic no argument constructor to the annotation and drop all arguments

@hrhino reminded us that on JDK 11+, Java's @Deprecated has optional since and forRemoval parameters. It would be good to keep the annotation even if some arguments cannot be interpreted.

[dwijnand]

• drop the annotation

I implemented this solution, using new JavaTokenData {}.copyFrom(in). Feedback welcome.

[lrytz]

The change to drop annotations looks good. Maybe add add a test that shows annotations are dropped in mixed compilation?

I also think an opt-in warning would be good.

Also, I was confused how references to Java compile-time constants work - now I remember, they are handled by the Java parser, which assigns a constant type to such fields where possible: #5487.

So unfortunately that means we need a way to silently drop annotatnions during type checking, for the case when the argument is not a compile-time constant:

public class C {
  public static @interface Ann {
    String value();
  }

  @Ann("a") public static int x0 = 0; // ok
  @Ann("a" + "b") public static int x1 = 1; // ok, dropped
  @Ann(C.konst) public static int x2 = 2; // ok
  @Ann(C.nokst) public static int x3 = 3; // error when type-checking

  public static final String konst = "muk";
  public static final String nokst = "muk" + "y";
}

with -Xprint:parser

    @new Ann(value = "a") <static> var x0: Int = _;
    <static> var x1: Int = _;
    @new Ann(value = C.konst) <static> var x2: Int = _;
    @new Ann(value = C.nokst) <static> var x3: Int = _;
    <static> val konst: String("muk") = _;
    <static> val nokst: String = _

and later

C.java:12: error: annotation argument needs to be a constant; found: C.nokst
  @Ann(C.nokst) public static int x3 = 3; // error when type-checking
         ^

[dwijnand]

Maybe add add a test that shows annotations are dropped in mixed compilation?

😕 That proved harder to do than expected (to me) using partest's existing machinery. The reason is it's not possible to have java sources that scalac compile and javac doesn't later compile (in this case putting the annotations that scalac dropped. The two options in partest are either:

1. scalac *.scala *.java and then javac *.java, the default with both file types, or
2. javac *.java only, by separating the java files into their own file "grouping", with a numeric suffix, like Foo_1.java.

Any suggestions? Lukas suggests @retronym or @hrhino might have a good suggestion, but also that there's no need to spend too much time on it.

I also think an opt-in warning would be good.

⚠️ Still TODO. I started looking around for where to define such an option and what to call it and spent about an hour reading through a bunch of settings code...

Suggestions (on both) welcome. 😄

So unfortunately that means we need a way to silently drop annotatnions during type checking, for the case when the argument is not a compile-time constant:

Done! ✅

[lrytz]

where to define such an option and what to call it

how about an -Xlint?

[dwijnand]

where to define such an option and what to call it

how about an -Xlint?

Discussed in the team today: decided not to add it for now (misc reasons: very niche, mostly @Deprecated, unsure if it should be Xlint, can't think of an obvious name).

So this is GTG now.

[som-snytt]

Missed opportunity for -Xlint:dwijnand. Anyway, it takes way longer than an hour to read through -Xlint code.

[lrytz]

Nice work! Thanks @dwijnand & @hrhino.

[retronym]

This triggered a spurious kind error with @Ann(List.class) in Java sources. Fixed in #8981

[retronym]

One more: #8982