Implicit capabilities not always applied/detected

[tbroyer]

Documentation (https://docs.gradle.org/5.6.4/userguide/managing_transitive_dependencies.html#sec:capabilities, https://docs.gradle.org/6.0/userguide/dependency_capability_conflict.html) says that “Each component defines an implicit capability corresponding to its GAV coordinates (group, artifact, version)” but this isn't actually always the case, or at least it doesn't always work to detect conflicts between dependencies.
I've only experienced it with log4j:log4j, but I doubt this is specific to that dependency/capability.

Expected Behavior

There should be a failure due to a conflict in capabilities.

Current Behavior

No conflict is detected, and both dependencies appear in the output (output of the dependencies and dependencyInsight tasks, and anything that lists or copies dependencies, e.g. building ZIPs, or Deb/RPM packages).

Context

I've noticed that our application ends up packaging several logging frameworks, including "conflicting" ones, such as jcl-over-slf4j and commons-logging, or log4j-over-slf4j and log4j (and log4j-1.2-api). Some of them were partially handled using exclusions (inherited from the time when the project was using Maven, before migrating it to Gradle), but that wasn't enough (and didn't resist badly handled upgrades that didn't update the exclusions).
I've started using capabilities to declare conflicts between those dependencies, and then capability resolution rules to resolve conflicts (we're using Log4j 2 through the SLF4J API). This worked well, except for one case: log4j:log4j vs. org.apache.logging.log4j:log4j-1.2-api.

Steps to Reproduce

Here's the simplest repro case I could come with:

plugins {
    java
}
repositories {
    mavenCentral()
}
dependencies {
    implementation("org.apache.logging.log4j:log4j-1.2-api:2.12.1")
    implementation("org.slf4j:slf4j-log4j12:1.7.29")

    components {
        withModule("org.apache.logging.log4j:log4j-1.2-api") {
            allVariants {
                withCapabilities {
                    addCapability("log4j", "log4j", id.version)
                }
            }
        }
    }
}

then run gradle dependencies --configuration=runtimeClasspath.

The bug won't manifest if declaring a direct dependency on log4j:log4j, only through transitive dependencies (in our project, log4j:log4 comes transitively from eu.medsea.mimeutil:mime-util in one subproject, and org.apache.logging.log4j:log4j-1.2-api as a direct dependency of another subproject, all being "assembled" in yet another subproject, possibly transitively through other subprojects).
It doesn't manifest either for org.slf4j:jcl-over-slf4j and commons-logging:commons-logging (when declaring that jcl-over-slf4j provides the commons-logging capability), or other similar conflicts.

Workarounds:

• explicitly declaring that log4j:log4j provides the log4j:log4j capability
• declaring that both components actually provide another capability (e.g. log4j:log4j-capability)
• declaring that log4j:log4j provides the org.apache.logging.log4j:log4j-1.2-api capability, rather than the reverse.

Your Environment

Build scan URL: https://scans.gradle.com/s/n5272o3emcjbs

[jjohannes]

Thanks for reporting @tbroyer. It indeed looks like a bug. The build scan also shows that two components with the same capability were selected:

• log4j has log4j:log4j
• log4j-1.2-api also has log4j:log4j

Another thing to observe is that we do not de-duplicate capabilities. In the seconds link you can see the log4j:log4j capability added twice. We should not add the same capability again if it already exists in a rule or let the implementations of MutableCapabilitiesMetadata implement set (instead of list) semantics (this might be unrelated to the issue here).

We will look into this.

[tbroyer]

I also noticed that accessing the capabilities will realize them and trigger the expected conflict:

dependencies {
    components {
        all {
            allVariants {
               withCapabilities {
                    capabilities.forEach {
                        println("$id -> ${it.group}:${it.name}:${it.version}")
                    }
                }
            }
        }
    }
}

or more simply:

dependencies {
    components {
        withModule("log4j:log4j") {
            withCapabilities {
                // do nothing, calling withCapabilities is enough to realize the implicit capabilities
            }
        }
    }
}

[tbroyer]

Fwiw, with Gradle 5.6, the only workaround I've found is to use a capability different from log4j:log4j (e.g. declare that log4j:log4j provides the org.apache.logging.log4j:log4j-1.2-api capability, or the cleaner alternative: declare that both log4j:log4j and org.apache.logging.log4j-log4j-1.2-api provide the log4j:log4j-capability capability)

[jjohannes]

Thanks for all the additional info @tbroyer. I am looking into this now and this seems to be an ordering issue concerning only implicit capabilities. So you can "fix" it by working only with explicit capabilities but also by order.

    implementation("org.apache.logging.log4j:log4j-1.2-api:2.12.1")
    implementation("org.slf4j:slf4j-log4j12:1.7.29")

In your sample, if you change the order of these two declarations it works. Which is bad (but the reason we did not catch this I suppose). Working on a fix now.

[jjohannes]

Fix is on master now and will be in the next 6.1 nightly.

Keeping this open in case there is a 6.0.1 - pick 50fdf24

[jjohannes]

Picked to release for 6.0.1