Improved file access logging

[johnkeates]

Is your proposal related to a problem?

When configuring rule files for the thanos ruler, the glob patterns need to be correct, but when they reference locations that are not accessible (i.e. no permissions) they fail silently since the only errors that would be caught are pattern errors.

Describe the solution you'd like

When trying to access a location that is not accessible, provide an option to log this, even if only at a debug log level.

Describe alternatives you've considered

I've considered an init script that pre-parses the globs and does all the checks ahead of time, but when running in a scratch container that means adding a script interpreter and a script, and doing it using a separate program seems redundant since this should be relatively safe to add to the existing code.

Additional context

I'm not sure if this is behaviour as intended, but can cause a lot of chasing down the wrong path when you (for example) have a CSI issue where a Pod should have access to a mounted volume but for some reason doesn't and your rule files aren't getting picked up (but no errors appear either).

Looking at

thanos/cmd/thanos/rule.go

Line 1008 in 6582c81

func reloadRules(logger log.Logger,

it seems that this is not something that Glob() can do directly as the comment already suggests, but some construction where the path is reduced to just before the first instance of a pattern could in theory be checked. Some JavaScript package seems to have the same idea (glob-parent) but it mostly relies on detecting the start of a pattern and slicing the path string at that location.