Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for java based compilers like Eclipse Compiler for Java (ecj) in Java toolchains #15942

Open
halexiev-concentric opened this issue Jan 26, 2021 · 11 comments
Labels
a:feature A new functionality in:toolchains Java Toolchains

Comments

@halexiev-concentric
Copy link

Provide support for Java based compilers like ecj in Java toolchains feature.

Expected Behavior

Should be able to configure a non distribution based compiler (non-executable) when provisioning a different compiler in Java toolchains feature.

Current Behavior

Only jdk executables (javac, etc.) distributions seem to be supported.

Context

Ecj allows compilation to proceed when there are minor and possible unrelated compile time errors from the code path one would like to execute. In a large multi-project build it is sometimes quite handy at development time to ignore such errors while somebody else is looking at them, for example. Javac does not allow compilation to proceed in case of any errors.

@jetztgradnet
Copy link

jetztgradnet commented Mar 16, 2022

I would also be interested in using ECJ for compiling with Gradle >= 7, as that allows makes hot-swapping much easier when building projects with Gradle on Java 17, but debugging them using Eclipse (and performing modifications while debugging).
See http://dplatz.de/blog/2018/java-hot-code-swapping.html for some hints wrt mixing compilers and the effects on hot swapping.

@errael
Copy link

errael commented Aug 24, 2022

Add frgaal, https://github.com/eppleton/frgaal, to the list of compilers that doesn't work with Gradle 7. Noting that here so a search picks it up.

@jjohannes
Copy link
Contributor

With some work done on the toolchain provisioning in Gradle 7.6, I had a fresh look at this topic. This is roughly what I think would be needed to make it work:

  • Context: The Eclipse Compiler is an alternative implementation of javax.tools.JavaCompiler. So this is about customising the compiler as part of a toolchain and not about exchanging the complete toolchain.
  • From that perspective, the first thing needed is the ability to construct a toolchain with the custom compiler included. One option could be that the JavaToolchainResolver somehow allows to transform/combine artifacts to a combined toolchain. E.g. download OpenJDK...tar.gz and repo1.maven.org/maven2/org/eclipse/jdt/ecj/3.26.0/ecj-3.26.0.jar and combine them. (One could work around this initially, by preparing a combined gzip file that contains everything and provide that through a custom repository/server.)
  • The second thing needed is that the additional classes with the custom compiler implementation (in this case ecj-3.26.0.jar) need to be added to the classpath of the compiler daemon worker. And we need to be able to tell Gradle to use a different compiler implementation. I think there unfortunately is no hook (e.g. some internal API) right now one could use to work around this.

@liblit
Copy link

liblit commented Nov 18, 2022

The T.J. Watson Libraries for Analysis, also known as WALA, builds using both the standard Java compiler and Eclipse's compiler. Our approach may be useful as an example of how to get this working within current Gradle limits.

This Groovy code implements a JavaCompileUsingEcj task type as a variant of the standard JavaCompile task. We do not create a completely new toolchain. Instead, we only customize the compiler-invocation behavior of a standard Java toolchain. This strategy is consistent with the approach that @jjohannes speculates about above:

this is about customising the compiler as part of a toolchain and not about exchanging the complete toolchain.

So to @jjohannes I'd say: yes, this is indeed a workable strategy.

@jjohannes also wrote:

The second thing needed is that the additional classes with the custom compiler implementation (in this case ecj-3.26.0.jar) need to be added to the classpath of the compiler daemon worker.

For WALA's JavaCompileUsingEcj, we use a detacted configuration to find, download, and cache ECJ. We then use that a bit later when putting together the compiler's command-line.

liblit added a commit to liblit/WALA that referenced this issue Feb 19, 2023
Gradle 8.0 includes Kotlin 1.8.0, and Kotlin 1.8.0 deprecates
`String.toLowerCase()`.  So in the `NullAway` script plugin, instead of
using `name.toLowerCase().contains("test")`, we now use
`name.contains("test", true)`.  The `true` argument requests
case-insensitive `String` comparison, which perfectly corresponds to
what the earlier code was doing anyway.

Gradle 8.0 adds an annoying extra check for `JavaCompile` tasks.  If
such a task is configured to fork a subprocess, Gradle 8.0 now requires
that the configured subprocess executable match the configured toolchain
compiler executable.  That's a problem for our ECJ Java compile tasks:
they want to fork a `java` subprocess, which does not match `javac` as
the toolchain compiler executable.  Working around this requires that we
adapt our `JavaCompileUsingEcj` task implementation *yet again*, as we
seemingly have to do with every new Gradle release, with
gradle/gradle#15942 still not implemented.  This time around, we avoid
relying on the standard `JavaCompile.compile()` task action method,
since calling that leads to the new executable equality check.  Instead,
we provide our own `compile()` task action method that directly runs the
ECJ subprocess with a suitable command line.

Along with that change, we also slightly change how we define and
resolve the ECJ jar.  Previously we used a detached `Configuration` for
this purpose, but that turns out to be a bit tricky to manage with the
Gradle configuration cache.  Now we use a normal, named `Configuration`
that is defined and configured in the shared `java.gradle.kts` script
plugin.

We also move the `skipJavaUsingEcjTasks` property check out of the
`JavaCompileUsingEcj` task constructor, because having it there produced
an IntelliJ IDEA warning about calling a non-`final` method in a
constructor.  Instead, we configure this check in an existing bit of
`java.gradle.kts` logic that was already being used to configure all
`JavaCompileUsingEcj` tasks.
liblit added a commit to liblit/WALA that referenced this issue Feb 19, 2023
Gradle 8.0 includes Kotlin 1.8.0, and Kotlin 1.8.0 deprecates
`String.toLowerCase()`.  So in the `NullAway` script plugin, instead of
using `name.toLowerCase().contains("test")`, we now use
`name.contains("test", true)`.  The `true` argument requests
case-insensitive `String` comparison, which perfectly corresponds to
what the earlier code was doing anyway.

Gradle 8.0 adds an annoying extra check for `JavaCompile` tasks.  If
such a task is configured to fork a subprocess, Gradle 8.0 now requires
that the configured subprocess executable match the configured toolchain
compiler executable.  That's a problem for our ECJ Java compile tasks:
they want to fork a `java` subprocess, which does not match `javac` as
the toolchain compiler executable.  Working around this requires that we
adapt our `JavaCompileUsingEcj` task implementation *yet again*, as we
seemingly have to do with every new Gradle release, with
gradle/gradle#15942 still not implemented.  This time around, we avoid
relying on the standard `JavaCompile.compile()` task action method,
since calling that leads to the new executable equality check.  Instead,
we provide our own `compile()` task action method that directly runs the
ECJ subprocess with a suitable command line.

Along with that change, we also slightly change how we define and
resolve the ECJ jar.  Previously we used a detached `Configuration` for
this purpose, but that turns out to be a bit tricky to manage with the
Gradle configuration cache.  Now we use a normal, named `Configuration`
that is defined and configured in the shared `java.gradle.kts` script
plugin.

We also move the `skipJavaUsingEcjTasks` property check out of the
`JavaCompileUsingEcj` task constructor, because having it there produced
an IntelliJ IDEA warning about calling a non-`final` method in a
constructor.  Instead, we configure this check in an existing bit of
`java.gradle.kts` logic that was already being used to configure all
`JavaCompileUsingEcj` tasks.
liblit added a commit to wala/WALA that referenced this issue Feb 20, 2023
Gradle 8.0 includes Kotlin 1.8.0, and Kotlin 1.8.0 deprecates
`String.toLowerCase()`.  So in the `NullAway` script plugin, instead of
using `name.toLowerCase().contains("test")`, we now use
`name.contains("test", true)`.  The `true` argument requests
case-insensitive `String` comparison, which perfectly corresponds to
what the earlier code was doing anyway.

Gradle 8.0 adds an annoying extra check for `JavaCompile` tasks.  If
such a task is configured to fork a subprocess, Gradle 8.0 now requires
that the configured subprocess executable match the configured toolchain
compiler executable.  That's a problem for our ECJ Java compile tasks:
they want to fork a `java` subprocess, which does not match `javac` as
the toolchain compiler executable.  Working around this requires that we
adapt our `JavaCompileUsingEcj` task implementation *yet again*, as we
seemingly have to do with every new Gradle release, with
gradle/gradle#15942 still not implemented.  This time around, we avoid
relying on the standard `JavaCompile.compile()` task action method,
since calling that leads to the new executable equality check.  Instead,
we provide our own `compile()` task action method that directly runs the
ECJ subprocess with a suitable command line.

Along with that change, we also slightly change how we define and
resolve the ECJ jar.  Previously we used a detached `Configuration` for
this purpose, but that turns out to be a bit tricky to manage with the
Gradle configuration cache.  Now we use a normal, named `Configuration`
that is defined and configured in the shared `java.gradle.kts` script
plugin.

We also move the `skipJavaUsingEcjTasks` property check out of the
`JavaCompileUsingEcj` task constructor, because having it there produced
an IntelliJ IDEA warning about calling a non-`final` method in a
constructor.  Instead, we configure this check in an existing bit of
`java.gradle.kts` logic that was already being used to configure all
`JavaCompileUsingEcj` tasks.
@TheMrMilchmann
Copy link
Contributor

This issue has become more apparent with Gradle 8 effectively dropping support for using custom compilers (viaforkOptions) for JavaCompile tasks. As it stands, there does not seem to be a way to use ecj instead of javac without rewriting significant parts of the Java plugins.

On a more general note, I'm somewhat annoyed by Gradle at this point as there seems to be a trend to remove old APIs before a (stable) replacement is available. Other example of this is the removal of the pluginBundle extension for the publishing plugin although the replacements are still incubating. I don't mind cleanup and like the general direction in which Gradle is heading, but it would be a shame to drop support for non-standard setups in the interim.

@jjohannes
Copy link
Contributor

There was a discussion about this in the Gradle Community Slack some time ago, where I also warned about being careful to not break this use case before there is a replacement:
https://gradle-community.slack.com/archives/CAHSN3LDN/p1670430803274029?thread_ts=1670406694.815029&cid=CAHSN3LDN (Links to Gradle community Slack)

My understanding was that setting the executable is still possible (more or less) as before. But I haven't tried myslef. Maybe @alllex can share here how this works now?

@liblit
Copy link

liblit commented Feb 21, 2023

The WALA project compiles all code twice: once using standard JavaCompile tasks and again using an ECJ-based JavaCompile task variant that we call JavaCompileUsingEcj. Each new Gradle release breaks the latter in some new way, and we figure out how to fix it again. If it would be helpful to anyone to see how we did it for Gradle 8.0, you can find the main implementation here.

That being said, it would be lovely if Gradle could just stop breaking this for us with each new release. If Gradle would support ECJ compilation directly, then that would be even better.

@jjohannes
Copy link
Contributor

Instead of making the Jar with the compiler part of the toolchain (which I considered in this comment #15942 (comment)), we could also think of as an add-on for the toolchain in the JavaCompile task. By providing an additional property – like customCompilerClasspath – through which users can provide such Jars.

Turned out this wouldn't need too much adjustments: #24649
(not tested intensively, but it looks as if this works)

@ljacomet
Copy link
Member

ljacomet commented May 4, 2023

@AceTheFace
Copy link

What's the status here? I'm quite confused. There are a lot of linked PRs around this topics, some seem to be merged, some have been closed, some are still open without progress since months. What are the plans to move forward here?

@jjohannes
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:feature A new functionality in:toolchains Java Toolchains
Projects
None yet
Development

No branches or pull requests

9 participants