[Thanos Storegateway ]"failed to read index-header from disk; recreating" path=/data/01H1K45K0FRT36S1RCAWHW7R9A/index-heade

[krimeshshah]

Thanos, Prometheus and Golang version used:

Object Storage Provider: AWS s3

What happened: We updated our thanos chart version from v15.0.3 to 15.0.4. After update we observed that thanos storegateway pod is not coming up. And it has these logs

What you expected to happen: Thanos storegateway pod should be up and running

How to reproduce it (as minimally and precisely as possible):

Full logs to relevant components: ts=2024-04-15T07:15:15.640299722Z caller=bucket.go:746 level=debug msg="loading new block" id=01H3VDS0JKWNQMQ3MN0EKJDG77 ts=2024-04-15T07:15:15.640342407Z caller=binary_reader.go:569 level=debug msg="failed to read index-header from disk; recreating" path=/data/01H3VDS0JKWNQMQ3MN0EKJDG77/index-header err="try lock file: open /data/01H3VDS0JKWNQMQ3MN0EKJDG77/index-header: no such file or directory" ts=2024-04-15T07:15:15.64043739Z caller=bucket.go:746 level=debug msg="loading new block" id=01H1K40KG2DG1JRJG23XNBMRR7 ts=2024-04-15T07:15:15.640458349Z caller=binary_reader.go:569 level=debug msg="failed to read index-header from disk; recreating" path=/data/01H1K40KG2DG1JRJG23XNBMRR7/index-header err="try lock file: open /data/01H1K40KG2DG1JRJG23XNBMRR7/index-header: no such file or directory" ts=2024-04-15T07:15:15.640571349Z caller=bucket.go:746 level=debug msg="loading new block" id=01H1K45K0FRT36S1RCAWHW7R9A ts=2024-04-15T07:15:15.641171169Z caller=binary_reader.go:569 level=debug msg="failed to read index-header from disk; recreating" path=/data/01H1K45K0FRT36S1RCAWHW7R9A/index-header err="try lock file: open /data/01H1K45K0FRT36S1RCAWHW7R9A/index-header: no such file or directory"

Anything else we need to know:

[MichaHoffmann]

That log is not problematic I think, does it log something else before it crashes?