Gradle 4.10 incorrect ordering between dependencies of dependent tasks

[alpar-t]

After upgrading to Gradle 4.10, the eclipse project files are no longer generated when running:

./gradlew eclipse

As a work-around we have to run:

./gradlew eclipse
./gradlew eclipseProject

We have this in the build script:

tasks.eclipse.dependsOn(cleanEclipse, copyEclipseSettings)

Expected Behavior

This used to work because all the dependencies of cleanEclipse were ran before all the dependencies of eclipse.
Implicit dependency should be implied between the task dependencies on each side.

Current Behavior

Tasks such as eclipseProject are ran before cleanEclipseProject as such the config files are note generated.

Context

Import projects to eclipse.

Steps to Reproduce (for bugs)

git clone git@github.com:elastic/elasticsearch.git
git checkout 9c541b8f729  # check out upgrade to Gradle 4.10 as we'll revert it on master
./gradlew eclipse
find -name .project | wc -l
# 14
./gradlew eclipseProject
find -name .project | wc -l
# 206

Your Environment

https://gradle.com/s/ud7wjtlzi4dss

[ljacomet]

I agree that this is a change of behavior, but your previous setup worked by accident.

Task.dependsOn(otherTask) only indicates that the task on the left must run after the task on the right. It says nothing about the order of running of the other dependencies or their dependencies.
Wiring a clean task before a real task is always tricky as you need to make sure no optimization or re-ordering of the graph can cause cleanup to happen out of order.

[alpar-t]

eclipse.dependsOn(cleanEclipse) is to be interpreted as eclipse needs to run after cleanEclipse, but wouldn't a task be completed when all the tasks it depends on are completed. So cleanEclipse would not be considered completed until all it's dependencies ran, and eclipse dependencies would have to wait for their parent as well. I always thought of "depends on" as "it and it's dependencies". Even if this was accidental there could be many build scripts out there relying on this , and I think it's behavior worth preserving as it makes it much easier to reason about the task execution graph.

[nedtwigg]

From the 4.10 userguide:

To completely rewrite existing Eclipse files, execute a clean task together with its corresponding generation task, like “gradle cleanEclipse eclipse” (in that order). If you want to make this the default behavior, add “tasks.eclipse.dependsOn(cleanEclipse)” to your build script. This makes it unnecessary to execute the clean task explicitly.

This does not work currently. Since gradle has recommended this practice for a long time, at the very least I think there ought to be a proactive warning to the user when the user attempts tasks.eclipse.dependsOn(cleanEclipse)

[alpar-t]

@nedtwigg In case it helps, one can enforce that ordering by being explicit about the tasks:

 tasks.eclipseProject.mustRunAfter tasks.cleanEclipseProject
  tasks.matching { it.name == 'eclipseClasspath' }.all {
    it.mustRunAfter { tasks.cleanEclipseClasspath }
  }
  tasks.matching { it.name == 'eclipseJdt' }.all {
    it.mustRunAfter { tasks.cleanEclipseJdt }
  }

[oehme]

I agree this should not be changed in a minor release, no matter if it conceptually makes sense or not, especially if we told users to make use of this approach.

[wolfs]

@atorok Did that work for Gradle 4.9?

[alpar-t]

@wolfs yes it did.

[wolfs]

This issue seems to be a duplicate of #2488. The issue seems to be happening more frequently with Gradle 4.10.

There have been no changes to task scheduling which I am aware of between 4.9 and 4.10.

[ghale]

This is actually not related to #2488 (or at least only remotely related). That issue deals with dependencies of the clean task in other projects allowing producer tasks in different projects to execute before the clean task. The issue here is the incomplete modeling of the dependency relationships between the clean and generation tasks. This just happened to work before because of the way the unrelated tasks were scheduled in the task graph (i.e. the clean tasks just happened to end up in there before the generation tasks). There were some changes to the dependency relationships in 4.10, however, that caused that ordering to change, resulting in the eclipseProject tasks showing up in the graph before the clean tasks (in the absence of any modeled relationship between the tasks).

To workaround this, we'll bake in the assumption that in the scenario that both clean and generation tasks are in the task graph together, you almost always want clean to run before generation. In the odd case that you might want clean to run after generation, you'll still be able to explicitly model that, but in the absence of anything like that, clean tasks will always run first.

[oehme]

@ghale Which changes caused this difference?

[big-guy]

@oehme Gary pointed me to 49596cd

This is the only production code change:
49596cd#diff-2e4ae82137db9f058ae9480610c5df44R107

[ghale]

The change in question was to ensure that eclipseProject tasks in included builds get run when the lifecycle task in the root build is executed. But the side effect is that those dependencies now end up getting added to the task graph before the dependencies of the clean task causing the difference in order.

[ghale]

The fix for this will be in the 4.10.1 release.

[oehme]

Thanks for the info and fix guys!