How to make update the configuration easier

[BraisGabin]

Expected Behavior

When I update detekt my configuration of detekt is updated too to reflect the new defaults/rules/flags.

Current Behavior

Right now detekt doesn't do that and it's a pain to keep it updated.

Context

• Update detekt and its configuration mozilla-mobile/android-components#11540 is a PR I made updating the detekt configuration. The work that I needed to do there to be sure that I update the new configuration but without loosing the previous configuration was huge. And, it's not clear if the previous documentation is the old default one or is a custom one so, maybe, it is using old default values.
• I usually use this script/algorithm to update my configuration: What is a recommended way of updating detekt config after a new release? #3558 (comment) but it uses generic merges that doesn't know about how yaml work so usually there are some conflicts. The good part of updating like this is that all the default values are updated correctly.
• Use the new configure config for formatting #4352 (comment) there are users that use this file https://github.com/detekt/detekt/blob/main/detekt-core/src/main/resources/default-detekt-config.yml to make diffs between it's configuration and the default one.

---

To be honest, I don't know how we can fix this issue. In my case I like to have ALL the configurations in my project so I can see all the flags of all the rules and configure it easily. I use the configuration like documentation or a documentation index. Using the script/algorithm linked in point 2 I can keep my documentation up to date but it's not as fast and easy as it should be. Problems:

• In the middle of the script you need to update manually the detekt version so it's easy to automatice. In my project I know where the version of detekt is defined so I could use sed. But that's not replicable to other projects.
• I'm merging two yamls but I'm not using the fact that they are yaml. So I get lots of conflicts that could be avoided if the merger would interpret the yaml instead of treting it as a plain file.

---

Any ideas about how to make it easier? Should we include a detektUpdate task on our gradle plugin? How could we implement it?

[3flex]

Just dropping this here: #3002 (comment)

IMHO this issue should be part of the broader discussion on how the configuration of detekt can be improved overall. Perhaps a config that allows for enabling all rules by default means the actual detekt config can be stripped back to either enable or disable app rules by default, then enabling or disabling individual rules as required.

That should avoid requiring any wholesale changes to the config file with new detekt releases.

[BraisGabin]

I'm late to the conversation, but if the old configuration method is being thrown out, this is a good opportunity to design the way configuration works from the ground up. From what I understand the main driver of this change is to get auto-complete in the IDE? This could also be achieved with JSON schema for YAML which can be achieved with zero config in IntelliJ. I can open another issue for that.

There are so many other possible areas of improvement for the config that I would want to see before a DSL.

I played with Ruby in the past and the Rubocop linter is extremely popular, with a very similar feature set to detekt. Looking at the configuration docs for that I see so many benefits of the approach they took (using YAML!) that solve some problems raised by detekt users.

Features I'd want from the new configurator (some already mentioned above in this thread) based on inspiration from Rubocop:

• Including/excluding files for all rules (Allow globally defined exclusion pattern #2581, Don't analyse Kotlin files generated by kapt detekt-compiler-plugin#10)

• Enable or disable all rules by default (failFast configuration #1528 (comment))

• Explicit inheritance rules. This makes things like multi-project configs in Gradle very simple - each project can have its own config which inherits from a root config, and this can be set within the config file and doesn't require any CLI flags or Gradle config (though that could also be implemented, as long as the order of inheritance were clearly defined)

• All config would always inherit from detekt's default config. Where this is not desired, user can just disable all rules by default within the config file then build their config from there (Align cli flags and defaults with other analysis tools #2267 (comment))

• Merging of hashes when configs are inherited, and user's choice of merging or overriding arrays when configs inherited (Merge (rather than replace) multiple configs for ForbiddenImport rule #2647)

• Walking the file tree to find a config file, and also checking user's home directories for a config in case there is no project-specific config

• CurrentIssues/ManuallySuppressedIssues file is replaced with just another config that's inherited from

• Choose a custom severity for an issue - not all issues are as important to all users (Severity level for rules #533 (comment))

• Setting compatibility for rules based on Kotlin version - e.g. if project targets Kotlin 1.3, a rule to add trailing commas for parameter lists etc. should not be suggested since it requires 1.4.

Not everything Rubocop does with its config can be literally copied for detekt as there are some differences in behaviour to consider for Java/Kotlin projects and Gradle, but majority of its config functionality could be implemented in detekt.

I've always found detekt's config setup generally confusing. I think this comes from incomplete documentation as well as config behaviour that's been built iteratively over time to fix individual issues or pain points without considering the overall picture. Detekt 2.0 is a good opportunity IMHO to rethink some of this and come up with something really robust that shouldn't need so much change over time.

Other point: Allow to instantiate the same rule multiple times.

To me this is a huge change that have a lot of requirements. How can we work on this to make it happen? I worked already by my own in the DSL or the refactor of the Rule and by extension the whole core of detekt but it is really difficult to finish it because it's huge. And the PRs are huge too and difficult to review so it's difficult to advance.

[BraisGabin]

We could also think about toml (I think that @JavierSegoviaCordoba told me this idea).

[3flex]

We could also think about toml

I personally don't think the config file format or DSL matters that much, though TOML might have properties that help with some of these goals (I haven't worked with it except for the Gradle versions catalog)

To me this is a huge change that have a lot of requirements. How can we work on this to make it happen?

It's true, there are a lot of suggestions for improvement. My point wasn't so much that this is the right end state, only that we should start with a design and/or spec, that addresses the issues with the current config and the features we will target. I'm sure most of it can be added iteratively. I just want to avoid adding more config features without a plan in place, because we may make things harder for ourselves in future.

[cortinico]

My 2 cents: I think a web tool could help here. I would fork this tool we use for React Native https://github.com/react-native-community/upgrade-helper

Essentially it allows you to diff two defined version of a library, to see what has changed. For example between 0.67.0 and 0.67.1: https://react-native-community.github.io/upgrade-helper/?from=0.67.0&to=0.67.1

We could adapt it to see how the default config file evolved (which at least for me is a bit of the pain point as the config file is rather big and hard to navigate sometimes).

As an improvement, we could extend the web tool to apply the resulting .patch file to a user's config file in an interactive manner.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Please comment or this will be closed in 7 days.

[BraisGabin]

How can we actionate this issue? Do we create a yaml differ and patcher (or find one)? Is it worth it? Are we going to continue working with yaml?

[igorwojda]

From a dev perspective, updating detect rules is always a pain - it is hard to keep custom detekt config up to date (to have all rules in the config file).

In ideal world I would like to see

1. A dedicated task that allows adding missing config options to the custom detekt config file (don't override existing rules and update detekt-formatting config as well)
2. A build warning or better custom failing detekt "config check rule" that would fail when a custom configuration does not contain all of the rules

Real-life scenario:

1. Manually upgrade detekt version
2. Run Gradle tasks to update config
3. Run detekt to verity
4. Create PR (now dev can. clearly see the diff so it is possible to make a decision whatever or not enable\disable each new rule)

[JavierSegoviaCordoba]

What about dropping configuration file types like Yaml and use Kotlin files/modules?

I was thinking how to argument this but with this is enough: Type safety.

Same problem GitHub Actions has and there is a Kotlin DSL with wrappers for common actions to generate the actions yaml files. Detekt wouldn't need to generate anything, just pick the list of rules provided via Kotlin with the kotlin type system.

[cortinico]

From a dev perspective, updating detect rules is always a pain - it is hard to keep custom detekt config up to date (to have all rules in the config file).

Have you considered using buildUponDefaultConfig?

What about dropping configuration file types like Yaml and use Kotlin files/modules?

In theory, I agree, it would be nice to add Kotlin support for config files (maybe support both)?
The reality is that the tooling for standalone Kotlin files is, IMHO, weaker than YAML.
Users will have a hard time configuring their IDE to have auto-completion support. I'm not sure if we want to go down this path.

[JavierSegoviaCordoba]

It should be a JVM module added as included build or added to the classpath so you can pass it to the Detekt block.

A JVM module is not too costly, I agree it is harder to create than just a yaml file but you get:

• Type safe API, probably Detekt can provide a nice DSL by adding as dependency a detekt-rule-builder or whatever.
• Unified way to create and provide rules.
• It is easy to share a Kotlin configuration because you can publish them to maven repositories and add them to any included build or the classpath, extend it with new rules or just add the rules to the Detekt block.

A lot of flexibility, unification, and safer way to work with Rules, but with a bit of overload (creating a module vs creating a file).

[cortinico]

A JVM module is not too costly, I agree it is harder to create than just a yaml file but you get:

side note that it comes will all the side effects of including dependencies that can clash with your Kotlin version + an included build increases your Gradle configuration time.

[BraisGabin]

I would go for something like *.detekt.kts. And an IDEA plugin to inject all the dependencies.

The problem of a new jvm module is that you can have a different configuration per module so it doesn't scale.

I would also like to have a dsl but I agree with @cortinico that it is a lot of tooling.

But this topic is really important. Our configuration isn't easy and that's not good to onboard new people.

[JavierSegoviaCordoba]

@BraisGabin you can already add dependencies to kts files.

The configuration itself should be the Gradle Detekt block, and the module/s itself should be a way to provide those configurations.

For example, it is up to the team the way they manage the that/those modules, at the end, in the plugin block, you should be adding the rules, for example:

Imagine that Detekt is setting rules.set(defaultRules) under the hood, then you want to add or change those rules.

detekt {
    rules.set(defaultRules + publishedCompanyModuleRules + localProjectModuleRules)
}

Similar behavior to Koin modules, if the same rule with a different value is provided two times, the latest has to preference and override the previous one.

[JavierSegoviaCordoba]

One really nice thing Detekt has, is being respectful with all or almost all Gradle best practices, which implies that projects are isolated, so it is the responsibility of a developer to provide the same rule sets to all modules in some way, probably via convention plugins as that is the recommended approach that the Gradle team is suggesting.

But it is up to them how they manage Gradle, and if they want the same rules in all projects for Detekt or whatever other plugin. I think it is not a Detekt responsibility to force them to share them in a specific way, it is a Gradle (or the build tool) responsibility.

Spotless is "similar" to Detekt and I think they haven't a problem with scaling and you have to provide the "rules" to each project spotless block.

[atulgpt]

Hi @BraisGabin I had a somewhat similar proposal at #6053 Even @cortinico suggestion at #6053 (comment) here valid and will work in streamlining the detekt defaults management

[BraisGabin]

To me the configuration of detekt is one of the major things that we should improve for 2.0 but I must say that I have no idea how to do it.

Problems:

• The configuration should be easy to find (autocomplete? Force the configuration to be always exhaustive?)
• On lists there should be an option to "modify" the default configuration. No just overwrite it.
• When updating detekt it should be easy to merge the current configuration with the new default configuration
• The configuration should allow to instantiate the same rule more than once with different configurations (This is not doable right now)

A possible solution is a DSL that generates a yml and then detekt reads that yml. I don't think that detekt itself should exeute a random script to get the configuration. The security implications of that would be huge.

[matejdro]

What if we add warnAboutMissingOptions: true flag to the config?

That way, whenever new rules are added, developer updating the plugin will get warned and can copy paste new rules from default config to the existing yml file. It is still a chore, but, assuming that there is fairly small amount of rules on each release, it sounds much easier than #3558 (comment) to me

EDIT: this appears to be in already as checkExhaustiveness, but it is not documented anywhere
EDIT2: but it does not appear to work for plugin rules, such as formatting, so it's not ideal