The exclude won't be able to exclude rewritten dependency

[attix-zhang]

Even if I add an exclude rule to exclude "com.google.guava:guava", if other dependencies are rewritten to "com.google.guava:guava", I will still get it in the dependency graph.

Take the following project for example:

plugins {
    id 'java'
}

repositories {
    mavenCentral()
}

configurations.all {
    resolutionStrategy.dependencySubstitution {
        substitute module("com.jcraft:jsch") using module("com.google.guava:guava:20.0") because "No reason, just for test"
    }
    exclude group: "com.google.guava", module: "guava"
}

dependencies {
    implementation group: 'org.apache.hadoop', name: 'hadoop-common', version: '2.6.0'
}

I already added an exclude rule: exclude group: "com.google.guava", module: "guava"

However, I still get guava in my dependency graph

Expected Behavior

If I exclude a dependency, I should not have it in my dependency graph anymore. No matter what.

Current Behavior

clone this repo: https://github.com/attix-zhang/test-gradle-exclude-with-substitution

and then run ./gradlew dependencyInsight --dependency com.google.guava:guava --configuration compileClasspath,

You will still get guava in the compileClasspath.

➜  test-gradle-exclude-with-substitution git:(main) ./gradlew dependencyInsight --dependency com.google.guava:guava --configuration compileClasspath

> Task :dependencyInsight
com.google.guava:guava:20.0 (selected by rule)
   variant "compile" [
      org.gradle.status              = release (not requested)
      org.gradle.usage               = java-api
      org.gradle.libraryelements     = jar (compatible with: classes)
      org.gradle.category            = library

      Requested attributes not found in the selected variant:
         org.gradle.dependency.bundling = external
         org.gradle.jvm.environment     = standard-jvm
         org.gradle.jvm.version         = 8
   ]

com.jcraft:jsch:0.1.42 -> com.google.guava:guava:20.0
\--- org.apache.hadoop:hadoop-common:2.6.0
     \--- compileClasspath

A web-based, searchable dependency report is available by adding the --scan option.

BUILD SUCCESSFUL in 1s

Context

Our company has some dependency substitution rules and every project is required to use. With this bug, each individual project is unable to exclude dependencies cleanly.

Steps to Reproduce

git clone https://github.com/attix-zhang/test-gradle-exclude-with-substitution.git
cd test-gradle-exclude-with-substitution
./gradlew dependencyInsight --dependency com.google.guava:guava --configuration compileClasspath

Your Environment

• Machine: Macbook Big Sur 11.5.1
• Java:

openjdk version "1.8.0_292"
OpenJDK Runtime Environment (Zulu 8.54.0.21-CA-macosx) (build 1.8.0_292-b10)
OpenJDK 64-Bit Server VM (Zulu 8.54.0.21-CA-macosx) (build 25.292-b10, mixed mode)

• Build scan URL: https://scans.gradle.com/s/23jewesslbjwq

[ljacomet]

This is indeed a consequence of the way excludes are processed in dependency graph resolution:

• The logic filters out dependencies according to excludes
• The remaining dependencies are checked for applicable substitutions

As a consequence:

• If you exclude foo and have a rule that replaces bar with foo, foo can appear in the results when your graph contains bar
• If you exclude foo and have a rule that replaces foo with bar, and nothing else brings in bar, then bar will not appear in the results

Ideally Gradle should fix that first use case while I believe the second one is an OK behavior.

For implementation details, check NodeState#visitDependencies.

Note that a workaround to fix this in your case is to make sure to exclude as well the source of the substitution.

[attix-zhang]

Thanks for the suggestions. However, it is almost impossible to add both exclude to thousands of projects.

Due to this issue, the central dependencies substation rule we had will cause a lot of projects at our company to break.
Would you mind prioritize this issue higher? Thanks a lot!

[attix-zhang]

Hi @ljacomet , I tried to submit a pull request to fix it issue, how could I get it reviewed from the Gradle community?

[ljacomet]

Thanks for submitting the PR, just started the tests and conversation over there.

[ljacomet]

Potential backport request for 6.9.x: #18163