[Bug]: IntelliJ dependencies on jars, not modules, due to shadow plugin

[kennknowles]

What happened?

Behavior observed: When control-clicking on a Java class name that goes between modules, I noticed it often went to a decompiled class file rather than the source.

I think I figured out the immediate cause of this: the dependencies in the IntelliJ module config are on the built jar, not on the other intelliJ module. Described at https://intellij-support.jetbrains.com/hc/en-us/community/posts/206224519-CTRL-clicking-goes-to-decompiled-class-instead-of-java-class

This also causes IntelliJ's "find usages" to miss usages, so a symbol appears unused in the project when it does have remaining uses.

I then think I figured out the further cause for our project: I think this is caused by depending on the shadow configuration. If we don't use the shadow plugin, the dependencies are loaded by IntelliJ normally.

For example, consider this set of deps from flink_runner.gradle:

compileOnly project(":sdks:java:build-tools")
implementation library.java.vendored_guava_32_1_2_jre
implementation project(path: ":sdks:java:core", configuration: "shadow")
implementation project(":runners:core-java")
implementation project(":runners:java-fn-execution")
implementation project(":runners:java-job-service")
implementation project(":sdks:java:extensions:google-cloud-platform-core")
implementation library.java.vendored_grpc_1_60_1
implementation library.java.slf4j_api
implementation library.java.joda_time
implementation library.java.args4j

• All the external dependencies are fine (in fact IntelliJ will correctly download sources for them).
• The dependencies on other modules default configuration also work as expected.
• The dependency on the core SDK shows up as a jar dependency, not a module dependency.

Additionally: the transitive Beam model dependencies also show up as jar dependencies. This could be due to being protos or it could be due to transitively going through a jar dependency. But it could also be due to the core SDK depending on the shadow configuration of the model protos. (I don't know what the shadow configuration even does for the protos).

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner