Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a utility in huggingface-cli to migrate the entire cache to a new location #2117

Open
sealad886 opened this issue Mar 15, 2024 · 1 comment

Comments

@sealad886
Copy link
Contributor

Is your feature request related to a problem? Please describe.
The cached blobs and various .json and other config files ensure the integrity of the cache. However, it wouldn't be unusual for someone to run out of space, need to replace a hard drive, or for any other reason move their cache. Right now, if that happens, you just need to start over and re-download all your files.

Describe the solution you'd like
Wouldn't it be great if the cache system could manage itself and figure out how to move everything over to a new home directory?

Describe alternatives you've considered
I've taken to symlinking out parts of the cache but using the default $HOME/.cache directory, so that the filepaths are the same but the physical data storage is granular enough for me.

Additional context

@Wauplin
Copy link
Contributor

Wauplin commented Mar 15, 2024

Hi @amcox886, the best way to set a different location for your cache is to set the HF_HOME environment variable. I don't think we will ever implement a way to "figure out how to move everything over to a new home directory". This should be decided by the user, given how sensitive it can be (and only the user knows what's best for their setup).

Regarding an utility to migrate the entire cache to a new location, that should be possible using mv on the entire cache folder (+ setting the env variable correctly). Unfortunately you might have problems with moving symlinks if volumes are differents/not formatted the same. We already have an issue for this: #1757.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants