test folder ignoring in config:base preset

[fgreinacher]

The config:base preset currently includes the following rules:

renovate/lib/config/presets/internal/default.ts

Lines 42 to 55 in e5ccf2b

	ignoreModulesAndTests: {
	description:
	'Ignore `node_modules`, `bower_components`, `vendor` and various test/tests directories',
	ignorePaths: [
	'**/node_modules/**',
	'**/bower_components/**',
	'**/vendor/**',
	'**/examples/**',
	'**/__tests__/**',
	'**/test/**',
	'**/tests/**',
	'**/__fixtures__/**',
	],
	},

At least test and tests are pretty much standard directory names, e.g. in .NET projects. In my opinion these should not be ignored, because you usually want to keep also your test frameworks up to date.

What are your thoughts behind this?

[rarkins]

I don't see a problem with it as I think this rule benefits the majority who don't want to update tests/ and is easily changed by the minority who do.

[fgreinacher]

What are your thoughts @Lorilatschki @ap0llo @dschafhauser?

[ap0llo]

I don't see a reason why not to update test dependencies. I haven't run into this issue personally, but I would have been surprised by this default setting and would have expected all project to be updated

[Lorilatschki]

@fgreinacher I argee what @ap0llo mentioned above. When I started configuring renovate I was also wondering why test dependencies are not updated. In my opinion I see also no reason why not to update test dependencies the same like the common once.

[BlairCurrey]

I don’t see a problem with it as I think this rule benefits the majority who don’t want to update tests/ and is easily changed by the minority who do.

What is the easy change? Ignore the ignorePaths preset and redeclare the ignorePaths with the desired item removed? That seems to be the way, although I was hoping for some way to effectively remove/override the specific item so that I don't have to now maintain this list of ignorePaths (from config:base in my case) going forward. I also found it problematic to even confirm that this was the problem - it wasn't until this post that I found what config:base consisted of. I do not see it here https://docs.renovatebot.com/presets-config/ nor did a config:base return any results in the docs.

I can understand not changing these defaults for backwards compatibility but i also expected the test directory to be included, and this is from the perspective of a TypeScript monolith.

[HonkingGoose]

I do not see it here https://docs.renovatebot.com/presets-config/ nor did a config:base return any results in the docs.

You're not seeing many hits for config:base because that's a old term. We use config:recommended starting from Renovate version 36.

[fgreinacher]

Just to see what happens to a more less standard .NET repository after removing the test entries:

All of these are useful upgrades to test dependencies.

@rarkins What do you think about my proposal above?

Could we make the ignorePaths language-specific? 🤔

[fgreinacher]

How about we add a message to the new PRs? I'm thinking of something like this maybe:

The big idea here is that we explain what we changed, and try to give people an easy way out of the new behavior.

I love this idea! Could be technically challenging though! Maybe we can somehow simplify to just indicate a major version bump N days after it has been rolled out?

[rarkins]

Users of the hosted app unfortunately don't have an easy way or learning or controlling major version bumps

[fgreinacher]

Yes, and I think @HonkingGoose proposal could improve this situation a little bit!

[rarkins]

To display that message in every PR just in case would be a mistake. It would only be feasible if we could somehow identify which users have a test/tests folder despite the fact they're currently ignored.

[fgreinacher]

It would only be feasible if we could somehow identify which users have a test/tests folder despite the fact they're currently ignored.

For this specific case yes!

But generally we could also have a tiny header in each section that goes something like this:

Renovate has recently been updated from version X to version Y. Click here to see the changelog.

Even if the breaking changes are not relevant it might be a good trigger to look at the changelog and maybe discover other helpful new features.

[fgreinacher]

I know this is very old, but I still think this is a wrong default behavior.

Not updating dependencies in these directories is comparable to not updating devDependencies for NPM packages. People can miss important updates to test frameworks, coverage tools, or security linters.

[rarkins]

It's not the default behavior: https://docs.renovatebot.com/configuration-options/#ignorepaths

It's part of config:recommended: https://docs.renovatebot.com/presets-config/#configrecommended but config:recommended is not the default preset either: https://docs.renovatebot.com/self-hosted-configuration/#onboardingconfig

It's the default behavior in the Mend hosted app, but users are free to override defaults. Self-hosted administrators are in complete control too.

Setting "defaults" (which is arguable here if it's justified to say it's even a default) is about trade-offs. Even if 90% of people prefer one way as default, you don't do that if that's going to be a problem for the other 10% and cause them to churn immediately due to misunderstandings or misgivings. In this case I think that 90% of people don't want us upgrading test fixtures, html files in test folders, etc - and they complained loudly at the time.

It could be interesting if we log all the ignored files though, so that at least people can see that specific files are ignored in debug logs

[fgreinacher]

Sorry, that was rushed a bit. Of course it's not the default, but I'd still assume that most folks use the recommended config, but I might of course be in my bubble here.

Good background knowledge with the complaints, thanks for sharing! I'm still wondering if that rule should be language-specific. Do you remember where the complaints came from back then? In NPM that might very well be a good choice. But as mentioned above typical .NET projects have their test projects (where the test/dev dependencies are stated) within a test/tests directory.

[rarkins]

Yes, it was perhaps even back when Renovate was npm-only. I'm open to pushing ignoredPaths to be manager-specific