Update build config to be more modular, using buildSrc convention plugins

[aSemy]

Is your feature request related to a problem? Please describe

Presently the Dokka Gradle config uses a lot of allprojects {} and subprojects {} blocks to share configuration, but this is discouraged by Gradle

• https://docs.gradle.org/current/userguide/sharing_build_logic_between_subprojects.html#sec:convention_plugins_vs_cross_configuration
• https://melix.github.io/blog/2021/12/composition-in-gradle.html

Using inheritance makes configuring the project difficult, as it prevents subprojects from being ordered nicely (for example, it's not easily possible to have 'empty' subprojects, as the subprojects{} block will add the plugins to them). It also means configuration is applied, even when not desired. For example, Dokka always applies the kotlin("jvm") plugin

dokka/build.gradle.kts

Line 44 in 8317436

plugin("org.jetbrains.kotlin.jvm")

But that's not required for the Gradle plugin project, as Gradle will supply the correct Kotlin plugin when the kotlin-dsl plugin is applied.

Describe the solution you'd like

Dokka uses buildSrc convention plugins to configure and apply configuration in a modular way.

• https://docs.gradle.org/current/samples/sample_convention_plugins.html
• https://docs.gradle.org/current/userguide/custom_plugins.html#sec:precompiled_plugins

Describe alternatives you've considered

Additional context

This would help with implementation of #2700, so the Gradle Plugin subproject would not have unecessary configuration applied to it (like the kotlin("jvm") plugin.

Are you willing to provide a PR?

I have made a start, but there's a lot of work to do!

[aSemy]

Other improvements I think can be made

• use the Java Test Fixtures plugin to share test-support code https://docs.gradle.org/current/userguide/java_testing.html#sec:java_test_fixtures
• Directly depending on another project's task can cause issues - this can be removed easily enough

  dokka/integration-tests/maven/build.gradle.kts

  Line 14 in c065412

  val setupMavenTask = project(":runners:maven-plugin").tasks.withType<SetupMaven>().single()

• The dependencies are a bit scattered. For example, JUnit Jupiter is manually defined as 5.6.0 in every reference, but some versions are defined in gradle.properties. Dependencies could be set in one place using Gradle Version Catalog, and aligned using a Gradle Version Platform https://docs.gradle.org/current/userguide/platforms.html#sub:using-platform-to-control-transitive-deps

[IgnatBeresnev]

@aSemy looks like the issue can be closed as completed? There's a PR for the module catalog, and integration tests I'd like to be refactored/re-written separately, there's a lot to improve..

[aSemy]

There's some more work to do to make the build modular, as there's still some cross-project dependencies (e.g. publishing to Maven in integration tests). #2704 really laid down the ground work, so fixing these problems won't take as long to fix. I'm going to wait until #2652 is done though.

Of course I don't need an issue to make PR, so this issue can be closed.

[aSemy]

The next few tasks I see are:

• Update some more Dokka properties to use the new DokkaBuildProperties

  • dokka_version

    dokka/build.gradle.kts

    Line 15 in 1acfb60

    val dokka_version: String by project

  • dokka_publication_channel and dokka_publication_channels - but @IgnatBeresnev can you confirm if these are still in use? I couldn't see where they are sourced in the project.
    

    dokka/build-logic/src/main/kotlin/org/jetbrains/DokkaPublicationChannel.kt

    Lines 40 to 41 in 1acfb60

    	val publicationChannel = this.properties["dokka_publication_channel"]?.toString()
    	val publicationChannels = this.properties["dokka_publication_channels"]?.toString()

  • Signing credentials
    

    dokka/build-logic/src/main/kotlin/org/jetbrains/publication.kt

    Lines 133 to 135 in 1acfb60

    	val signingKeyId: String? = System.getenv("SIGN_KEY_ID")
    	val signingKey: String? = System.getenv("SIGN_KEY")
    	val signingKeyPassphrase: String? = System.getenv("SIGN_KEY_PASSPHRASE")

  • Space repo credentials
    

    dokka/build-logic/src/main/kotlin/org/jetbrains/publication.kt

    Lines 59 to 60 in 1acfb60

    	username = System.getenv("SPACE_PACKAGES_USER")
    	password = System.getenv("SPACE_PACKAGES_SECRET")

• Improve the cross-project publication dependencies required for the integration tests (related to Integration tests refactoring #2925)

  • Remove:
    

    dokka/build-logic/src/main/kotlin/org/jetbrains/taskUtils.kt

    Lines 6 to 14 in 1acfb60

    	fun Task.dependsOnMavenLocalPublication() {
    	project.rootProject.allprojects.forEach { otherProject ->
    	otherProject.invokeWhenEvaluated { evaluatedProject ->
    	evaluatedProject.tasks.findByName("publishToMavenLocal")?.let { publishingTask ->
    	this.dependsOn(publishingTask)
    	}
    	}
    	}
    	}

  • Replace with https://github.com/adamko-dev/dev-publish-plugin

• Tidy up Dokka's publication config. This is a more significant task, and it's more difficult to test.

  A lot of the config in build-logic/src/main/kotlin/org/jetbrains/publication.kt can be cleaned up, for example

  • Some of this config (like conditionally registering a repo) can be converted and simplified to use the example Restricting publications to specific repositories
  • There's probably no need for publication names to be distinct - the publication names are just an internal Gradle identifier
    

    dokka/build-logic/src/main/kotlin/org/jetbrains/publication.kt

    Line 29 in 1acfb60

    register<MavenPublication>(publicationName) {

• Replace the cross-project task fetching in plugins/base/build.gradle.kts with cross-project sharing

[aSemy]

I'm puzzled about what to do with ValidatePublications. It needs an update (it uses cross-project config so it's not compatible with project isolation, and it uses project at runtime which isn't compatible with Configuration Cache).

I can see the value of a test to make sure that publications are configured correctly, especially since there are multiple repos that need to be enabled/disabled conditionally based on the Dokka version and publication type, but I'm not sure how to implement that using a Gradle task...

[IgnatBeresnev]

@aSemy I'll explain how it works and what I think it was made for, maybe it'll give you some ideas.

In TeamCity we have the following configuration parameters for publishing:

Where the dokka_version dropdown contains:

• snapshot
• dev
• publish
• rc

Based on the chosen dokka_version, it adds a suffix to dokka_version_number, and then passes the resulting variable as the dokka_version Gradle property, which is used for publishing (and which is validated). Note: the publish type adds no suffix.

So if dokka_version == dev && dokka_version_number == 1.8.20, it will run the build with dokka_version = 1.8.20-dev.

The validate publication task asserts that the chosen dokka_publication_channels can accept the dokka_version that was passed.

So that we don't end up publishing 1.8.20-dev to Maven Central, or 1.8.20-SNAPSHOT to Space.

Note: the gradle-plugin-portal is broken atm, it doesn't work. There's a separate checkbox "Gradle plugin portal upload" - if it's set, TeamCity runs the publishPlugins task first, and only then dokkaPublish

---

Whether that's something that we need is a good question... It's nice to have, but if it's very difficult to implement or support, we can discuss dropping it.

checkProjectDependenciesArePublished() - this check also seems nice, but I wonder if it's needed or if there's a better way to do that? From what I understand, it checks that all of the internal project dependencies are published to a repository. I'm sure it must be a common concern in libraries?

[aSemy]

Thanks for the info @IgnatBeresnev. I've been digging around in the code and seeing the intended usage makes it more clear.

Something else to note is that ValidatePublications was created before the integration tests that use :publishToMavenLocal.

Current goal

My current goal is to refactor the build logic to make Dokka build config one step closer to config cache compatibility, which is required for updating to Gradle 8.

Proposed actions

Here's a list of my proposed actions:

• make the build config tidier by moving the following into DokkaBuildProperties
  • publicationChannels
  • the signing credentials
  • and dokka_version
• Remove dokka_publication_channel - from what you describe it's unused
• Update the 'version X should only be published to repo Y' logic to use an onlyIf {} check on the publishing tasks, as described in the Restricting publications to specific repositories Gradle docs
  • Remove the Space Dokka Dev repo conditional logic
• Replace fun Task.dependsOnMavenLocalPublication() to make the build compatible with Project Isolation (proposed replacement: https://github.com/adamko-dev/dev-publish-plugin/)

---

checkProjectDependenciesArePublished() - this check also seems nice, but I wonder if it's needed or if there's a better way to do that? From what I understand, it checks that all of the internal project dependencies are published to a repository. I'm sure it must be a common concern in libraries?

It's a really nice idea, but it would need a big re-write to be compatible with the newer Gradle restrictions.

However, I don't think it's necessary any more. The check was written before Dokka had any integration tests. This is no longer the case, and Dokka verifies publications are published by using MavenLocal. So, I think it can be safely removed!

Here are two ideas I have to re-write the test to be compatible. Both of these would be complicated and redundant, but they're fun to think about.

• Each published subproject also produces a "I've been published with $group:$artifact:$version" report file (similar to the frontend files are shared).

  Other projects would fetch these files, and confirm each of their Dokka-subproject runtime dependencies has provided a report file, with the correct GAV.

• Create a 'test-build-infra' subproject that uses Gradle TestKit to load Dokka, and run publishing tasks, and verify the results.