Move custom gradle configs into custom plugins #721

m273d15 · 2019-10-14T10:56:11Z

Moves our custom configs of IDE specific gradle plugins in custom plugins
which are written in Java. This reduces the usage of implicit groovy/gradle magic,
allows to reduce the amount of code in the build.gradle files and allows to reuse more code.
Removes the mavenize workflow that uses an existing eclipse installation as P2 Repository and
converts this repository into a maven repository. The new workflow uses the maven artifacts which
are deployed into maven central by the eclipse community. The mapping of eclipse version + bundle id -> maven artifact is performed by the goomph mavencentral plugin.
Adds a workaround for duplicate classpath entries in the eclipse build path.

Also solves the issue #517 in the intellij environment.

m273d15 · 2019-10-14T13:24:31Z

TODO: Remove workaround that removes duplicate classpath entries. Already fixed with #722

tobous

Review in progress. But here are the first notes.

buildSrc/src/main/java/saros/gradle/eclipse/SarosEclipseExtension.java

buildSrc/build.gradle

buildSrc/src/main/java/saros/gradle/intellij/ConfigureIntelliJTask.java

buildSrc/src/main/java/saros/gradle/intellij/SarosIntellijExtension.java

buildSrc/src/main/java/saros/gradle/intellij/ConfigureIntelliJTask.java

buildSrc/src/main/java/saros/gradle/intellij/SarosIntellijExtension.java

buildSrc/src/main/java/saros/gradle/intellij/ConfigureIntelliJTask.java

tobous · 2019-10-15T15:42:51Z

I think Codacy is a bit broken. The first analysis failed to find anything (as it did not analyze the 'buildSrc' directory), and the results of the successful analysis were not posted here. Anyways, @m273d15, you should have a look at the results of the newest codacy build.

tobous

Ok, I only had a very coarse look at the changes for the Eclipse build stuff as I am not really familiar with it or the Eclipse usage (as I don't use Eclipse myself). Also, the review already took me long enough as it is.
So it would be nice if somebody else could have a more detailed look at the eclipse (and STF) part of the code as well.

In general, a lot of classes and methods (at least for the IntelliJ stuff) need javadoc. There are a lot of implicit assumptions based on "insider knowledge" about Gradle, Gradle Tasks, IntelliJ, and the IntelliJ Gradle build process that is not mentioned at all. This makes it a lot harder to understand the logic.

buildSrc/src/main/java/saros/gradle/intellij/ConfigureIntelliJTask.java

build.gradle

buildSrc/build.gradle

buildSrc/src/main/java/saros/gradle/eclipse/SarosEclipseExtension.java

buildSrc/src/main/java/saros/gradle/eclipse/SarosEclipsePlugin.java

stf/build.gradle

m273d15

Wow, thank you for your review 👍

build.gradle

m273d15 · 2019-10-17T12:01:54Z

build.gradle

@@ -123,37 +108,16 @@ configure(projectsToConfigure) { projectToConf ->
    from projectToConf.configurations.releaseDep
  }

-  projectToConf.ext.eclipseHome = projectToConf.hasProperty('eclipseHome') ? eclipseHome : System.getenv('ECLIPSE_HOME')


It means that the developers don't have to use a local eclipse installation in order to build the eclipse plugin. You still need a local installation in order to run the plugin (which would be the next step of the infrastructure improvement). This has the advantage that you don't need to install eclipse if you want to build all projects in your ide without problems.
If you want to use a specific eclipse version for build and testing you can set the value:

sarosEclipse { eclipseVersion = <version string> }

Is this due to complications in the eclipse build process that required the "Mavenize" job in the previous setup?

The previous setup was necessary, because the eclipse bundles were not available via maven central, but the bundle are now deployed into maven central (by the eclipse project itself). Therefore, we don't need the weird setup of converting an eclipse installation (more specific the P2 repository) into a local maven repository. The only bundle that is not available via maven central is gef (because it is not part of the default bundles)

buildSrc/build.gradle

buildSrc/src/main/java/saros/gradle/eclipse/SarosEclipseExtension.java

buildSrc/src/main/java/saros/gradle/intellij/SarosIntellijPlugin.java

stf/build.gradle

tobous · 2019-10-28T14:15:42Z

Also, since this (at least slightly) changes the necessary build setup (e.g. the reference to a local eclipse installation no longer being necessary), the documentation should be updated accordingly.

buildSrc/src/main/java/saros/gradle/intellij/SarosIntellijPlugin.java

buildSrc/src/main/java/saros/gradle/intellij/IntellijConfigurator.java

buildSrc/src/main/java/saros/gradle/intellij/SarosIntellijExtension.java

buildSrc/src/main/java/saros/gradle/intellij/SarosIntellijPlugin.java

buildSrc/src/main/java/saros/gradle/eclipse/SarosEclipsePlugin.java

docs/contribute/development-environment.md

intellij/build.gradle

tobous · 2019-10-31T16:28:39Z

buildSrc/src/main/java/saros/gradle/intellij/IntellijConfigurator.java

+import org.jetbrains.intellij.IntelliJPluginExtension;
+
+/** Gradle task that configures the intellij plugin. */
+public class IntellijConfigurator {


Why does this no longer extend DefaultTask? Is it no longer necessary?

buildSrc/src/main/java/saros/gradle/intellij/IntellijConfigurator.java

intellij/build.gradle

stefaus

eclipse and IntelliJ (runIde) worked for me, but gradlew stfTest failed with:

exception

> Task :saros.stf.test:compileStfTestJava FAILED
[gitfolder]/stf.test/test/saros/stf/test/followmode/SimpleFollowModeITest.java:8: error: package org.eclipse.jface.bindings.keys does not exist
import org.eclipse.jface.bindings.keys.IKeyLookup;
                                      ^
[gitfolder]/stf.test/test/saros/stf/test/filefolderoperations/FileOperationsTest.java:11: error: package org.eclipse.core.runtime does not exist
import org.eclipse.core.runtime.CoreException;
                               ^
[gitfolder]/stf.test/test/saros/stf/test/filefolderoperations/FileOperationsTest.java:194: error: cannot find symbol
  public void testNewPkgAndClass() throws CoreException, IOException {
                                          ^
  symbol:   class CoreException
  location: class FileOperationsTest
[gitfolder]/stf.test/test/saros/stf/test/editing/ConcurrentEditingTest.java:8: error: package org.eclipse.jface.bindings.keys does not exist
import org.eclipse.jface.bindings.keys.IKeyLookup;
                                      ^
[gitfolder]/stf.test/test/saros/stf/test/invitation/ParallelInvitationWithTerminationByHostTest.java:13: error: package org.eclipse.core.runtime does not exist
import org.eclipse.core.runtime.CoreException;
                               ^
[gitfolder]/stf.test/test/saros/stf/test/invitation/ParallelInvitationWithTerminationByHostTest.java:65: error: cannot find symbol
      throws IOException, CoreException, InterruptedException {
                          ^
  symbol:   class CoreException
  location: class ParallelInvitationWithTerminationByHostTest
[gitfolder]/stf.test/test/saros/stf/test/invitation/HostInvitesBelatedlyTest.java:8: error: package org.eclipse.core.runtime does not exist
import org.eclipse.core.runtime.CoreException;
                               ^
[gitfolder]/stf.test/test/saros/stf/test/followmode/SimpleFollowModeITest.java:55: error: cannot find symbol
        .pressShortcut(IKeyLookup.BACKSPACE_NAME, IKeyLookup.BACKSPACE_NAME);
                       ^
  symbol:   variable IKeyLookup
  location: class SimpleFollowModeITest
[gitfolder]/stf.test/test/saros/stf/test/followmode/SimpleFollowModeITest.java:55: error: cannot find symbol
        .pressShortcut(IKeyLookup.BACKSPACE_NAME, IKeyLookup.BACKSPACE_NAME);
                                                  ^
  symbol:   variable IKeyLookup
  location: class SimpleFollowModeITest
[gitfolder]/stf.test/test/saros/stf/test/editing/ConcurrentEditingTest.java:72: error: cannot find symbol
    ALICE.remoteBot().editor(FILE).pressShortcut(new String[] {IKeyLookup.BACKSPACE_NAME});
                                                               ^
  symbol:   variable IKeyLookup
  location: class ConcurrentEditingTest
[gitfolder]/stf.test/test/saros/stf/test/editing/ConcurrentEditingTest.java:78: error: cannot find symbol
    ALICE.remoteBot().editor(FILE).pressShortcut(new String[] {IKeyLookup.BACKSPACE_NAME});
                                                               ^
  symbol:   variable IKeyLookup
  location: class ConcurrentEditingTest
11 errors

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':saros.stf.test:compileStfTestJava'.
> Compilation failed; see the compiler error output for details.

stefaus · 2019-11-12T22:09:25Z

buildSrc/src/main/java/saros/gradle/eclipse/configurator/OsgiDependencyConfigurator.java

+    return Arrays.stream(value.split(","))
+        .map(
+            (it) -> {
+              return it.split(";")[0];
+            })
+        .filter(
+            (it) -> {
+              return !bundlesToExclude.contains(it);
+            })
+        .toArray(String[]::new);


nit:

return Arrays.stream(value.split(",")) .map(it -> it.split(";")[0]) .filter(it -> !bundlesToExclude.contains(it)) .toArray(String[]::new);

m273d15 · 2019-11-13T12:36:38Z

eclipse and IntelliJ (runIde) worked for me, but gradlew stfTest failed with:

I know it is a simple fix. Seems like I forgot to add this as comment (I commented this issue somewhere ...). I will add the fix and the remaining improvements as soon as possible.

buildSrc/src/main/java/saros/gradle/eclipse/configurator/OsgiDependencyConfigurator.java

buildSrc/src/main/java/saros/gradle/eclipse/configurator/JarConfigurator.java

tobous · 2019-11-19T13:14:13Z

docs/contribute/development-environment.md

-* You need an **Intellij IDEA** installation which is used for dependency resolution. Install a current version of [**IntelliJ IDEA**](https://www.jetbrains.com/idea/download/#section=linux) (we have only tested Saros/I for IntelliJ releases 2017.X and later)
-
-Set the **system-wide environment variable `ECLIPSE_HOME`** to the eclipse installation dir that contains the directory `plugins`.<br/>
+* *Optional:* You can install an **Intellij IDEA** installation which is used for dependency resolution. Install a current version of [**IntelliJ IDEA**](https://www.jetbrains.com/idea/download/#section=linux) (we have only tested Saros/I for IntelliJ releases 2017.X and later)<br/>
 Set the **system-wide environment variable `INTELLIJ_HOME`** to the intellij installation dir that contains the directory `lib`.


Suggested change

Set the **system-wide environment variable `INTELLIJ_HOME`** to the intellij installation dir that contains the directory `lib`.

Set the **system-wide environment variable `INTELLIJ_HOME`** to the IntelliJ installation dir that contains the directory `lib`.

* *Optional:* You can also set the **system-wide environment variable `SAROS_INTELLIJ_SANDBOX`** to specify the base directory in which the IntelliJ sandboxes will be created. Otherwise, the directory `intellij/build` in the repository will be used by default.

docs/contribute/development-environment.md

m273d15 · 2019-11-19T14:30:45Z

Fixed all mentioned issues. @tobous Sorry, the review board is very chaotic. Seems like I missed the issues you already highlighted in the previous review.

saros-infrastructure · 2019-11-19T14:33:12Z

Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- buildSrc/src/main/java/saros/gradle/intellij/SarosIntellijExtension.java  1
- buildSrc/src/main/java/saros/gradle/eclipse/SarosEclipseExtension.java  1
- buildSrc/src/main/java/saros/gradle/eclipse/SarosEclipsePlugin.java  4
- buildSrc/src/main/java/saros/gradle/eclipse/configurator/EclipseConfigurator.java  1
- buildSrc/src/main/java/saros/gradle/eclipse/configurator/OsgiDependencyConfigurator.java  7
- buildSrc/src/main/java/saros/gradle/intellij/SarosIntellijPlugin.java  3
- buildSrc/src/main/java/saros/gradle/eclipse/configurator/JarConfigurator.java  3
- buildSrc/src/main/java/saros/gradle/intellij/IntellijConfigurator.java  6

See the complete overview on Codacy

tobous · 2019-11-19T14:34:23Z

docs/contribute/development-environment.md

-* You need an **IntelliJ IDEA** installation which is used for dependency resolution. Install a current version of [**IntelliJ IDEA**](https://www.jetbrains.com/idea/download/) (we have only tested Saros/I for IntelliJ releases 2017.X and later)
-
-Set the **system-wide environment variable `ECLIPSE_HOME`** to the Eclipse installation dir that contains the directory `plugins`.<br/>
+* *Optional:* You can install an **IntelliJ IDEA** installation which is used for dependency resolution. Install a current version of [**IntelliJ IDEA**](https://www.jetbrains.com/idea/download/#section=linux) (we have only tested Saros/I for IntelliJ releases 2017.X and later).<br/>


I think this change got lost during rebasing (or rather the code suggestion this change is based on was still based on the old content of the file).

Suggested change

* *Optional:* You can install an **IntelliJ IDEA** installation which is used for dependency resolution. Install a current version of [**IntelliJ IDEA**](https://www.jetbrains.com/idea/download/#section=linux) (we have only tested Saros/I for IntelliJ releases 2017.X and later).<br/>

* *Optional:* You can install an **IntelliJ IDEA** installation which is used for dependency resolution. Install a current version of [**IntelliJ IDEA**](https://www.jetbrains.com/idea/download/) (we have only tested Saros/I for IntelliJ releases 2017.X and later).<br/>

Add the SarosEclipsePlugin that configures the gradle eclipse plugin and provides methods to create osgi-bundles and resolve bundle dependencies from MANIFEST.MF This plugin uses the goomph mavencentral plugin in order to match eclipse bundle ids to maven artifacts. This makes the mavenize step obsolete Add the SarosIntellijPlugin that configures the gradle intellij plugin and enhances the runIde task by locking/unlocking actions that allow us to use muliple simultaneously running runIde instances (as already configured in the intellij/build.gradle).

The gef entry is superfluous since the stf is in a separate project and bundle. Therefore, the entry has to be removed.

All other bundles can be resolved by the goomph plugin (which is used in the custom saros gradle eclipse plugin). This is not an eclipse core bundle and therefore not deployed to maven central.

m273d15 · 2019-11-21T08:19:41Z

rebased, removed #section=linux

m273d15 added the Area: Infrastructure Issue affecting the infrastructure to build, test and deploy Saros and its documentation label Oct 14, 2019

m273d15 requested review from tobous and stefaus October 14, 2019 10:56

m273d15 force-pushed the pr/impr_gradle branch from c9df221 to dcaff4f Compare October 14, 2019 11:05

tobous requested changes Oct 14, 2019

View reviewed changes

tobous requested changes Oct 15, 2019

View reviewed changes

m273d15 mentioned this pull request Oct 17, 2019

task buildSearchableOptions is executed by default in intellij #725

Closed

m273d15 commented Oct 17, 2019

View reviewed changes

tobous reviewed Oct 28, 2019

View reviewed changes

buildSrc/src/main/java/saros/gradle/intellij/SarosIntellijPlugin.java Outdated Show resolved Hide resolved

m273d15 force-pushed the pr/impr_gradle branch 2 times, most recently from 213a724 to 72599a2 Compare October 31, 2019 14:19

saros-project deleted a comment Oct 31, 2019

tobous requested changes Oct 31, 2019

View reviewed changes

tobous reviewed Nov 1, 2019

View reviewed changes

intellij/build.gradle Outdated Show resolved Hide resolved

m273d15 mentioned this pull request Nov 7, 2019

Failing STF tests on master #744

Closed

stefaus reviewed Nov 12, 2019

View reviewed changes

m273d15 force-pushed the pr/impr_gradle branch 2 times, most recently from 6e4b6ba to b50160d Compare November 19, 2019 12:29

m273d15 requested a review from tobous November 19, 2019 12:41

tobous requested changes Nov 19, 2019

View reviewed changes

tobous reviewed Nov 19, 2019

View reviewed changes

docs/contribute/development-environment.md Outdated Show resolved Hide resolved

m273d15 force-pushed the pr/impr_gradle branch from b50160d to 448d15f Compare November 19, 2019 14:28

m273d15 requested a review from tobous November 19, 2019 14:31

tobous requested changes Nov 19, 2019

View reviewed changes

tobous previously approved these changes Nov 20, 2019

View reviewed changes

m273d15 added 7 commits November 21, 2019 09:17

[BUILD] Add new buildSrc project to intellij config

1560ec6

[BUILD] Remove gef from eclipse manifest

5f3f9ab

The gef entry is superfluous since the stf is in a separate project and bundle. Therefore, the entry has to be removed.

[BUILD] Add gef libs as workaround for whiteboard

52957d2

All other bundles can be resolved by the goomph plugin (which is used in the custom saros gradle eclipse plugin). This is not an eclipse core bundle and therefore not deployed to maven central.

[BUILD] Define gradle property defaults

eb08c19

[BUILD] Use custom gradle plugins

1257dde

[DOC] update development environment docs

fbfb1b6

m273d15 dismissed tobous’s stale review via fbfb1b6 November 21, 2019 08:18

m273d15 force-pushed the pr/impr_gradle branch from 448d15f to fbfb1b6 Compare November 21, 2019 08:18

tobous approved these changes Nov 21, 2019

View reviewed changes

m273d15 merged commit d125f9d into master Nov 21, 2019

m273d15 deleted the pr/impr_gradle branch November 21, 2019 13:17

tobous mentioned this pull request Nov 21, 2019

Whiteboard causes build to fail randomly #769

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move custom gradle configs into custom plugins #721

Move custom gradle configs into custom plugins #721

m273d15 commented Oct 14, 2019

m273d15 commented Oct 14, 2019

tobous left a comment

tobous commented Oct 15, 2019

tobous left a comment

m273d15 left a comment

m273d15 Oct 17, 2019

tobous commented Oct 28, 2019

tobous Oct 31, 2019

stefaus left a comment

stefaus Nov 12, 2019

m273d15 commented Nov 13, 2019

tobous Nov 19, 2019 •

edited

m273d15 commented Nov 19, 2019

saros-infrastructure commented Nov 19, 2019

tobous Nov 19, 2019 •

edited

m273d15 commented Nov 21, 2019

	Set the system-wide environment variable `INTELLIJ_HOME` to the intellij installation dir that contains the directory `lib`.
	Set the system-wide environment variable `INTELLIJ_HOME` to the IntelliJ installation dir that contains the directory `lib`.
	* Optional: You can also set the system-wide environment variable `SAROS_INTELLIJ_SANDBOX` to specify the base directory in which the IntelliJ sandboxes will be created. Otherwise, the directory `intellij/build` in the repository will be used by default.

	* Optional: You can install an IntelliJ IDEA installation which is used for dependency resolution. Install a current version of [IntelliJ IDEA](https://www.jetbrains.com/idea/download/#section=linux) (we have only tested Saros/I for IntelliJ releases 2017.X and later).<br/>
	* Optional: You can install an IntelliJ IDEA installation which is used for dependency resolution. Install a current version of [IntelliJ IDEA](https://www.jetbrains.com/idea/download/) (we have only tested Saros/I for IntelliJ releases 2017.X and later).<br/>

Move custom gradle configs into custom plugins #721

Move custom gradle configs into custom plugins #721

Conversation

m273d15 commented Oct 14, 2019

m273d15 commented Oct 14, 2019

tobous left a comment

Choose a reason for hiding this comment

tobous commented Oct 15, 2019

tobous left a comment

Choose a reason for hiding this comment

m273d15 left a comment

Choose a reason for hiding this comment

m273d15 Oct 17, 2019

Choose a reason for hiding this comment

tobous commented Oct 28, 2019

tobous Oct 31, 2019

Choose a reason for hiding this comment

stefaus left a comment

Choose a reason for hiding this comment

stefaus Nov 12, 2019

Choose a reason for hiding this comment

m273d15 commented Nov 13, 2019

tobous Nov 19, 2019 • edited

Choose a reason for hiding this comment

m273d15 commented Nov 19, 2019

saros-infrastructure commented Nov 19, 2019

tobous Nov 19, 2019 • edited

Choose a reason for hiding this comment

m273d15 commented Nov 21, 2019

tobous Nov 19, 2019 •

edited

tobous Nov 19, 2019 •

edited