maven_install does not differentiate compile and runtime scopes, causing circular dependencies

[ejona86]

This causes circular dependencies when processed by Bazel, but are processed without issue with Maven and Gradle. The contents here are mostly taken from grpc/grpc-java#10576 (comment) , so see it for more context. The correct thing to do is for rules_jvm_external to use runtime_deps, but jvm_import doesn't support runtime deps. (It's unclear to me why all deps in jvm_import can't be treated as runtime_deps. Maybe something involved with the hjar. In Maven Central, compile scope normally behaves as exports in Bazel as hjars aren't available in Gradle/Maven (see also #147), and runtime scope is like Bazel's runtime_deps.)

In gRPC 1.58.0 the grpc-core artifact had part of its contents moved into a new grpc-util artifact. To preserve existing behavior, grpc-core added a runtime-only dependency on grpc-util. grpc-util has a compile-time dependency on grpc-core.

io.grpc:grpc-core:1.58.0's POM:

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <groupId>io.grpc</groupId>
  <artifactId>grpc-core</artifactId>
  <version>1.58.0</version>
  <!-- ... -->
  <dependencies>
    <!-- ... -->
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-util</artifactId>
      <version>1.58.0</version>
      <scope>runtime</scope>
    </dependency>
  </dependencies>
</project>

io.grpc:grpc-util:1.58.0's POM:

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <groupId>io.grpc</groupId>
  <artifactId>grpc-util</artifactId>
  <version>1.58.0</version>
  <!-- ... -->
  <dependencies>
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-core</artifactId>
      <version>1.58.0</version>
      <scope>compile</scope>
    </dependency>
    <!-- ... -->
  </dependencies>
</project>

With this WORKSPACE, Bazel finds "cycle in dependency graph":

# rules_jvm_external boilerplate
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

RULES_JVM_EXTERNAL_TAG = "5.3"

RULES_JVM_EXTERNAL_SHA = "d31e369b854322ca5098ea12c69d7175ded971435e55c18dd9dd5f29cc5249ac"

http_archive(
    name = "rules_jvm_external",
    sha256 = RULES_JVM_EXTERNAL_SHA,
    strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
    url = "https://github.com/bazelbuild/rules_jvm_external/releases/download/%s/rules_jvm_external-%s.tar.gz" % (RULES_JVM_EXTERNAL_TAG, RULES_JVM_EXTERNAL_TAG),
)

load("@rules_jvm_external//:repositories.bzl", "rules_jvm_external_deps")

rules_jvm_external_deps()

load("@rules_jvm_external//:setup.bzl", "rules_jvm_external_setup")

rules_jvm_external_setup()
# END rules_jvm_external boilerplate

load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
    artifacts = [
        "io.grpc:grpc-okhttp:1.58.0",
    ],
    repositories = [
        "https://repo.maven.apache.org/maven2/",
    ],
)

$ bazel build @maven//:io_grpc_grpc_okhttp
ERROR: CACHE/3b4b3cfdd7a4382a6a990a3244c39990/external/maven/BUILD:243:11: in jvm_import rule @maven//:io_grpc_grpc_util: cycle in dependency graph:
    @maven//:io_grpc_grpc_okhttp (442d8db79c046018027f86fb6ba2e9e3560c56c0504c8b6423d08c7d06207c4d)
.-> @maven//:io_grpc_grpc_util (442d8db79c046018027f86fb6ba2e9e3560c56c0504c8b6423d08c7d06207c4d)
|   @maven//:io_grpc_grpc_core (442d8db79c046018027f86fb6ba2e9e3560c56c0504c8b6423d08c7d06207c4d)
`-- @maven//:io_grpc_grpc_util (442d8db79c046018027f86fb6ba2e9e3560c56c0504c8b6423d08c7d06207c4d)
ERROR: Analysis of target '@maven//:io_grpc_grpc_okhttp' failed; build aborted
INFO: Elapsed time: 0.339s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (45 packages loaded, 391 targets configured)

Looking at the generated BUILD file, rules_jvm_external doesn't use runtime_deps for the runtime-scoped:

jvm_import(
        name = "io_grpc_grpc_core",
        jars = ["v1/https/repo.maven.apache.org/maven2/io/grpc/grpc-core/1.58.0/grpc-core-1.58.0.jar"],
        deps = [
                ":com_google_android_annotations",
                ":com_google_code_gson_gson",
                ":com_google_errorprone_error_prone_annotations",
                ":com_google_guava_guava",
                ":io_grpc_grpc_api",
                ":io_grpc_grpc_context",
                ":io_grpc_grpc_util",
                ":io_perfmark_perfmark_api",
                ":org_codehaus_mojo_animal_sniffer_annotations",
        ],
        tags = [
                "maven_coordinates=io.grpc:grpc-core:1.58.0",
                "maven_url=https://repo.maven.apache.org/maven2/io/grpc/grpc-core/1.58.0/grpc-core-1.58.0.jar",
        ],
        visibility = ["//visibility:public"],
)

And that's because jvm_import doesn't support runtime_deps, unlike java_import.

[nickbreen]

In addition to runtime there is a case for provided dependencies too.

E.g. org.clojure:spec.alpha and org.clojure:clojure

Though it seems in this case it might just be a specific subset of versions that have the cycle.

[tmccombs]

Is there any workaround for this?

[vorburger]

Is there any workaround for this?

Sometimes you can workaround using exclusions, like in grpc/grpc-java#10576.

[nickbreen]

To avoid having to wade through that rather long issue the relevant bit is:

maven_install(
    artifacts = [
        "io.grpc:grpc-okhttp:1.58.0",
        maven.artifact(
            artifact = "grpc-core",
            exclusions = [
                "io.grpc:grpc-util",
            ],
            group = "io.grpc",
            version = "1.58.0",
        ),
    ],
    repositories = [
        "https://repo.maven.apache.org/maven2/",
    ],
)

[nickbreen]

It looks like support for deps/runtime_deps/provided_deps is non-trivial.

1. lock file JSON will need to differentiate the dep's scope (likely requiring a v3 lock file format) and have compile_dependencies , runtime_dependencies, and provided_dependencies keys or similar. Note that V2's dependencies is the union of the three new keys' values.
2. pinning will need to generate the new lock file with the new keys
3. consuming the lock file and ultimately generating the java_library rules needs to use new lock file and new keys.

[nickbreen]

Looks like there's also a dependency on coursier's behaviour: coursier/coursier#1231

Rather, the json report from coursier is used: it does not differentiate dependencies' scopes.

[manjutapali]

I am facing similar issue while trying to update the version of google-cloud-storage package.

I have tried the following suggestion as well, still it is failing.

        maven.artifact(
            artifact = "google-cloud-storage",
            exclusions = [
                "io.grpc:grpc-context",
            ],
            group = "com.google.cloud",
            version = "2.30.1",
        ),

[agluszak]

If you're using 5.3, that's because this commit is not included. I've filed a request to release a new version: #1014