possible issue with filelock/.locks folder

[grahamannett]

Describe the bug

I am not sure how to reproduce this bug but I have gotten it 2x now this year and not clear why. Also not entirely clear if it is from huggingface_hub but it seems like it would be as presumably nothing else is messing with these files/folders and this issue is specific to the .locks folder.

Here is what I see when the .locks folder seems to get some error for example trying to download a new model:

>>> from transformers import AutoTokenizer, AutoModelForCausalLM
>>>
>>> tokenizer = AutoTokenizer.from_pretrained("openai-community/gpt2")
Traceback (most recent call last):
  File "/home/graham/mambaforge/envs/tr/lib/python3.11/pathlib.py", line 1116, in mkdir
    os.mkdir(self, mode)
OSError: [Errno 74] Bad message: '/home/graham/.cache/huggingface/hub/.locks/models--openai-community--gpt2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/graham/mambaforge/envs/tr/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 737, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/graham/mambaforge/envs/tr/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 569, in get_tokenizer_config
    resolved_config_file = cached_file(
                           ^^^^^^^^^^^^
  File "/home/graham/mambaforge/envs/tr/lib/python3.11/site-packages/transformers/utils/hub.py", line 389, in cached_file
    resolved_file = hf_hub_download(
                    ^^^^^^^^^^^^^^^^
  File "/home/graham/mambaforge/envs/tr/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/graham/mambaforge/envs/tr/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1421, in hf_hub_download
    Path(lock_path).parent.mkdir(parents=True, exist_ok=True)
  File "/home/graham/mambaforge/envs/tr/lib/python3.11/pathlib.py", line 1125, in mkdir
    if not exist_ok or not self.is_dir():
                           ^^^^^^^^^^^^^
  File "/home/graham/mambaforge/envs/tr/lib/python3.11/pathlib.py", line 1250, in is_dir
    return S_ISDIR(self.stat().st_mode)
                   ^^^^^^^^^^^
  File "/home/graham/mambaforge/envs/tr/lib/python3.11/pathlib.py", line 1013, in stat
    return os.stat(self, follow_symlinks=follow_symlinks)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 74] Bad message: '/home/graham/.cache/huggingface/hub/.locks/models--openai-community--gpt2'

The errors are specific to this .locks folder and online info seems to say use fsdk or similar tools but since it is the same drive that has the OS it is easier to just move the whole hub folder to a different place/drive and then move the models* and similar from hub over to a newly generated .cache/huggingface/hub folder if that makes sense.

I am unable to do anything with this .locks folder though, e.g.

$ ls /home/graham/.cache/huggingface/hub/.locks/
ls: reading directory '/home/graham/.cache/huggingface/hub/.locks/': Bad message
total 0

$ sudo rm -rf /home/graham/.cache/huggingface/hub/.locks/
rm: cannot remove '/home/graham/.cache/huggingface/hub/.locks/': Directory not empty

It might be too hard to pinpoint what is causing this.

Reproduction

No response

Logs

No response

System info

Copy-and-paste the text below in your GitHub issue.

- huggingface_hub version: 0.19.4
- Platform: Linux-6.5.0-21-generic-x86_64-with-glibc2.35
- Python version: 3.11.6
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /home/graham/.cache/huggingface/token
- Has saved token ?: True
- Who am I ?: besiktas
- Configured git credential helpers: store
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.1.2
- Jinja2: 3.1.2
- Graphviz: N/A
- Pydot: N/A
- Pillow: 10.2.0
- hf_transfer: N/A
- gradio: 4.16.0
- tensorboard: N/A
- numpy: 1.26.4
- pydantic: 2.5.3
- aiohttp: 3.9.1
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /home/graham/.cache/huggingface/hub
- HF_ASSETS_CACHE: /home/graham/.cache/huggingface/assets
- HF_TOKEN_PATH: /home/graham/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10