[nix-cache] Allow use of multiple caches. Split read vs write #2059
Conversation
internal/boxcli/cache.go
Outdated
| permissions := cache.GetPermissions() | ||
| fmt.Fprintf( | ||
| cmd.OutOrStdout(), | ||
| "* %s (read: %t, write %t)\n", |
There was a problem hiding this comment.
Minor suggestion - instead of printing the bools we could make it a little more human-readable with something like:
* s3://private-cache
* s3://devbox-cache (read-only)
* https://cache.nixos.org (read-only)
| // Token expiration should always be 60 minutes, but we want to expire | ||
| // the cache a bit earlier to avoid having a stale token. | ||
| // Adding a 40 minute check to ensure we don't accidentally create a | ||
| // very short-lived token. | ||
| if time.Until(exp) > 40*time.Minute { | ||
| exp = exp.Add(-10 * time.Minute) | ||
| } |
There was a problem hiding this comment.
This changes the filecache expiration, but not the credential expiration. Should the buffer be the same for both?
filecacheexpiration determines whendevbox cache credentialswill make an API request for a new token.AWSCredentials.Expirationdetermines when the AWS C++ SDK will calldevbox cache credentialsagain to get a new token. It caches the token in-memory so that it doesn't need to invoke the credential process before every request.- The AWS SDK doesn't write the token to disk anywhere, so the cached in-memory token is only valid for the duration of the Nix process or the token expiration (whichever happens first). After that, it calls
devbox cache credentialsagain.
There was a problem hiding this comment.
if the aws c++ sdk respects the expiration then this change is not needed. Do you know if that's the case if the expiration happens in the middle of a command?
My concern was that in long running jobs the token might expire in the middle of the job. by adding this 10 minute buffer it guarantees that devbox cache credentials will return something that has at least 10 minutes left. Maybe it's not needed? I can remove.
There was a problem hiding this comment.
I believe it should respect the expiration (at least that's what the aws help config-vars docs make it sound like. I haven't tested it though.
I'm wondering what happens if the nix daemon gets a token and then tries to use it close to the expiration time. The daemon is obviously long-lived, so it will probably have that token right up until the expiration. Does the AWS CLI/SDK already have a small buffer to account for clock skew? If not, maybe we should still include a small one.
Co-authored-by: Greg Curtis <greg.curtis@jetpack.io> Signed-off-by: Mike Landau <mikeland86@gmail.com>
Summary
How was it tested?
Untested, needs new api to be deployed