Allow persisting repository cache after uploading to S3

[PhilipAbed]

Tell us more.

The current behavior is to not save repository cache to disk when uploading to S3.

I would like to allow saving the repository cache so i can use it after the run, and then delete it manually.
I noticed that the repository cache file path is implemented differently in S3 and local.

local: saves the cache under {cacheDir}/renovate/repository/{platform}/{organization}/{repository}.json
s3: saves the cache under s3://{Bucket/subDirectory}/{platform}/{organization}/{repository}/cache.json

since the file name is not consistent and i don't want to change the current behaviour.
I suggest adding new self hosted config option repositoryCachePersist,
This flag is related to S3 only currently,
when enabled, it will persist the cache locally
there are 2 options

1. it can be a boolean
   if it's true, then it will copy the local.ts implementation, and save file locally as well.
   this means that the path to the file will be the same as if we set repositoryCacheType = local (default) in addition to uploading to s3.
   if its false then the repository cache will be uploaded to S3 and will not be persisted in local cache.
2. user can choose where to persist it,
   the flag will have 3 options: disabled, s3, local
   by default it will be disabled,
   if set to s3, we will add it to the same path as s3 under cacheDir.
   if set to local we will add it to the same path as local under cacheDir

[nabeelsaabna]

what about a simpler solution that is same behavior for both local and s3
add a new flag repositoryCacheMakeACopy = /path/to/copy.json
and it can be implemented in the base implementation

[secustor]

I would see this as an array.
For writing all endpoints would be written.
On Read the first which successful returns is used.

That way you get the cache written to both AND we can have remote and local cache simultanios.

[rarkins]

From the Mend app point of view, all we need is an option to "save a copy locally even if pushing to s3". I'm not sure we need the extra complexity unless others also need this

[rarkins]

I'd like to do this:

• New experimental flag RENOVATE_X_REPO_CACHE_S3_LOCAL
• If set, a copy of written to local too even if S3 is configured
• Renovate still reads from S3 first

[secustor]

If we follow that behaviour I rather use RENOVATE_X_REPO_CACHE_FORCE_LOCAL mentioning both S3 and local in the same variable wouldn't tell me what it is actually doing.

[PhilipAbed]

@rarkins if you would like to change the flag name, please comment in Pull Request on the final name

also i prefer if the flag was controlled in config, can it not be experimental?

[rarkins]

If you need it merged soon, please stick with experimental