Regression in 5.5 when using dependency constraints for non-jar dependencies without a POM

[pkoenig10]

Expected Behavior

Using dependency constraints to resolve a non-jar dependency without a POM file should succeed.

Current Behavior

Using dependency constraints to resolve a non-jar dependency without a POM file fails because Gradle expected to find either a POM file or a jar file.

Context

This works on version 5.4.1. It stopped working in version 5.5-rc-1 and continues to not work up through version 5.6.2.

I was able to narrow down the regression further by testing against snapshots. This works in 5.5-20190515000032+0000 but does not work in 5.5-20190515115345+0000.

We are using Gradle to resolve and download dependencies that were not published with Gradle/Maven. More specifically, we are using Gradle to resolve and download a archive with static files that we want to include the jar we publish.

Steps to Reproduce

Here is a minimal build.gradle file necessary to reproduce the issue:

repositories {
    mavenCentral()
    maven {
        url 'https://dl.bintray.com/palantir/releases'
    }
}

configurations {
    pomWithVersion
    pomWithConstraint
    noPomWithVersion
    noPomWithConstraint
}

// These are unrealistic dependencies, but they just examples to demonstrate the issue
dependencies {
    pomWithVersion 'com.palantir.conjure:conjure:4.6.2@tgz'
    pomWithConstraint 'com.palantir.conjure:conjure@tgz'

    noPomWithVersion 'jruby:jruby-bin:0.9.9@distribution-tgz'
    noPomWithConstraint 'jruby:jruby-bin@distribution-tgz'

    constraints {
        pomWithConstraint('com.palantir.conjure:conjure:4.6.2')
        noPomWithConstraint('jruby:jruby-bin:0.9.9')
    }
}

task resolvePomWithVersion {
    doLast {
        configurations.pomWithConstraint.resolve()
    }
}

task resolvePomWithConstraint {
    doLast {
        configurations.pomWithConstraint.resolve()
    }
}

task resolveNoPomWithVersion {
    doLast {
        configurations.noPomWithVersion.resolve()
    }
}

task resolveNoPomWithConstraint {
    doLast {
        configurations.noPomWithConstraint.resolve()
    }
}

resolvePomWithVersion, resolvePomWithConstraint, and resolveNoPomWithVersion succeed on all versions.

resolveNoPomWithConstraint succeeds on version 5.4.1 and fails on versions 5.5-rc-1 and later with the following error:

> Task :resolveNoPomWithConstraint FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/path/to/project/build.gradle' line: 49

* What went wrong:
Execution failed for task ':resolveNoPomWithConstraint'.
> Could not resolve all files for configuration ':noPomWithConstraint'.
   > Could not find jruby:jruby-bin:0.9.9.
     Searched in the following locations:
       - https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.pom
       - https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar
       - https://dl.bintray.com/palantir/releases/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.pom
       - https://dl.bintray.com/palantir/releases/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar
     Required by:
         project :
   > Could not find jruby:jruby-bin:0.9.9.
     Searched in the following locations:
       - https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.pom
       - https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar
       - https://dl.bintray.com/palantir/releases/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.pom
       - https://dl.bintray.com/palantir/releases/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar
     Required by:
         project :

This fails even if I add transitive = false to the dependency.

Be sure to run these tasks with --refresh-dependencies to ignore cached dependencies downloaded when using Gradle versions without the bug.

Your Environment

I don't think there's anything specific in my environment that would affect this issue. But I'm happy to provide build scans or more info if you are unable to reproduce the issue and/or would find them useful.

[pkoenig10]

I have some cleaned up debug logs that pretty clearly show the regression.
debug-5.4.1.log
debug-5.5.log

If you look at the diff, on 5.4.1 we use the correct distribution-tgz extension while on 5.5 we use jar.

$ diff debug-5.4.1.log debug-5.5.log
134c134
< [LIFECYCLE] [class org.gradle.internal.buildevents.TaskExecutionLogger] > Task :resolveNoPomWithConstraint
---
> [LIFECYCLE] [class org.gradle.internal.buildevents.TaskExecutionLogger] > Task :resolveNoPomWithConstraint FAILED
157,165c157,165
< [DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.DefaultExternalResourceArtifactResolver] Loading https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Metadata of https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz' started
< [DEBUG] [org.gradle.internal.resource.transport.http.HttpResourceAccessor] Constructing external resource metadata: https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz
< [DEBUG] [org.gradle.internal.resource.transport.http.HttpClientHelper] Performing HTTP HEAD: https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Completing Build operation 'Metadata of https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz'
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Metadata of https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz' completed
< [DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver] Found artifact but no meta-data for module 'jruby:jruby-bin:0.9.9' in repository 'maven', using default meta-data.
< [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.modulecache.PersistentModuleMetadataCache] Recording module descriptor in cache: jruby:jruby-bin:0.9.9 [changing = false]
< [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainComponentMetaDataResolver] Using jruby:jruby-bin:0.9.9 from Maven repository 'maven'
---
> [DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.DefaultExternalResourceArtifactResolver] Loading https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar
> [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Metadata of https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar' started
> [DEBUG] [org.gradle.internal.resource.transport.http.HttpResourceAccessor] Constructing external resource metadata: https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar
> [DEBUG] [org.gradle.internal.resource.transport.http.HttpClientHelper] Performing HTTP HEAD: https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar
> [INFO] [org.gradle.internal.resource.transport.http.HttpClientHelper] Resource missing. [HTTP HEAD: https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar]
> [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Completing Build operation 'Metadata of https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar'
> [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Metadata of https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar' completed
> [DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver] No meta-data file or artifact found for module 'jruby:jruby-bin:0.9.9' in repository 'maven'.
> [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.modulecache.PersistentModuleMetadataCache] Recording absence of module descriptor in cache: jruby:jruby-bin:0.9.9 [changing = false]
170,171d169
< [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder] Visiting configuration jruby:jruby-bin:0.9.9(default).
< [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder] jruby:jruby-bin:0.9.9(default) has no transitive incoming edges. ignoring outgoing edges.
174,189d171
< [DEBUG] [org.gradle.internal.component.model.ComponentAttributeMatcher] Selected match jruby:jruby-bin:0.9.9 from candidates [jruby:jruby-bin:0.9.9] for {}
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Resolve files of :noPomWithConstraint' started
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Resolve jruby-bin.distribution-tgz (jruby:jruby-bin:0.9.9)' started
< [DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.DefaultExternalResourceArtifactResolver] Loading https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz
< [DEBUG] [org.gradle.internal.resource.transfer.DefaultCacheAwareExternalResourceAccessor] Constructing external resource: https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Metadata of https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz' started
< [DEBUG] [org.gradle.internal.resource.transport.http.HttpResourceAccessor] Constructing external resource metadata: https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz
< [DEBUG] [org.gradle.internal.resource.transport.http.HttpClientHelper] Performing HTTP HEAD: https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Completing Build operation 'Metadata of https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz'
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Metadata of https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz' completed
< [INFO] [org.gradle.internal.resource.transfer.DefaultCacheAwareExternalResourceAccessor] Cached resource https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.distribution-tgz is up-to-date (lastModified: Mon Sep 09 13:31:59 PDT 2019).
< [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository] Downloaded artifact 'jruby-bin.distribution-tgz (jruby:jruby-bin:0.9.9)' from resolver: maven
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Completing Build operation 'Resolve jruby-bin.distribution-tgz (jruby:jruby-bin:0.9.9)'
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Resolve jruby-bin.distribution-tgz (jruby:jruby-bin:0.9.9)' completed
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Completing Build operation 'Resolve files of :noPomWithConstraint'
< [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationExecutor] Build operation 'Resolve files of :noPomWithConstraint' completed
197,198c179,206
< [LIFECYCLE] [org.gradle.internal.buildevents.BuildResultLogger]
< [LIFECYCLE] [org.gradle.internal.buildevents.BuildResultLogger] BUILD SUCCESSFUL in 1s
---
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] FAILURE: Build failed with an exception.
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Where:
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Build file '/Users/pkoenig/Downloads/test/build.gradle' line: 51
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * What went wrong:
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Execution failed for task ':resolveNoPomWithConstraint'.
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] > Could not resolve all files for configuration ':noPomWithConstraint'.
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]    > Could not find jruby:jruby-bin:0.9.9.
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]      Searched in the following locations:
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]        - https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.pom
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]        - https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]      Required by:
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]          project :
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]    > Could not find jruby:jruby-bin:0.9.9.
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]      Searched in the following locations:
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]        - https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.pom
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]        - https://repo.maven.apache.org/maven2/jruby/jruby-bin/0.9.9/jruby-bin-0.9.9.jar
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]      Required by:
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]          project :
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Try:
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Run with --stacktrace option to get the stack trace.  Run with --scan to get full insights.
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
> [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Get more help at https://help.gradle.org
> [ERROR] [org.gradle.internal.buildevents.BuildResultLogger]
> [ERROR] [org.gradle.internal.buildevents.BuildResultLogger] BUILD FAILED in 4s

[jjohannes]

@pkoenig10 thanks for the detailed report and for taking the time to narrow this down. Unfortunately, I cannot reproduce it. Running your minimal build does not produce the issue for me with any of the indicated versions. It's very suspicious. Did you run the build in isolation or is there maybe anything else in your settings that might cause a problem?
I also took a look at the code that is responsible for doing the artifact selection. Nothing has changed there between the versions you indicated. So it must be a side effect of something else.

Can you check again and maybe run your minimal build with an isolated Gradle user home (--gradle-user-home) and check if it still fails? And if yes, could you provide a build scan?

[pkoenig10]

I am able to reproduce this issue with an isolated Gradle user home.

I was also able to reproduce this issue using the official Gradle docker image. I ran the following command:

docker run --rm -v $PWD/build.gradle:/home/gradle/build.gradle gradle:5.5-jdk11 gradle resolveNoPomWithConstraint --scan

With the following build.gradle file (Github doesn't support uploading .gradle files):
build.gradle.zip

The build scan for this build can be found here:
https://scans.gradle.com/s/rwfmch3eopark

Note that I was also able to reproduce this issue using all of the following official Gradle docker images:

• 5.5-jre8
• 5.5-jre11
• 5.5-jdk8
• 5.5-jdk11

[pkoenig10]

I was able to get a successful using Gradle 5.4.1 build by running the same command with the same build.gradle file:

docker run --rm -v $PWD/build.gradle:/home/gradle/build.gradle gradle:5.4.1-jdk11 gradle resolveNoPomWithConstraint --scan

And here's the build scan for that build:
https://scans.gradle.com/s/ff2qnnz24v5y2

[jjohannes]

@pkoenig10 thanks again for taking the time and the hint of using the docker images. Running in a clean container exposed the issue. Which, as it turned out, is a more involved issue with discovering a component without metadata in a repository - see #10996 for details. The issue was hidden if an earlier version of Gradle already discovered the component. In that case, we do not check the repository again as we already know that the component exists there.

The fix #10996 will be in Gradle 6.0. Feel free to try the next 6.0 nightly (6.0-20191010... or higher) to confirm that it fixes the issue for you.
Please note that in Gradle 6.0+ you need to set artifact() metadata source to make Gradle look for modules without metadata - see upgrading guide. In your example, it could look like this::

    mavenCentral() {
	metadataSources { artifact() }
    }
    maven {
        url 'https://dl.bintray.com/palantir/releases'
    }

We are considering another patch release for 5.6 and will possibly include this fix. I'll keep this issue open until that is decided/done.

[ljacomet]

Closing as this was merged for 6.0 and 5.6.3