Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add bodaay/HuggingFaceModelDownloader #19791

Conversation

CrystalMethod
Copy link
Contributor

bodaay/HuggingFaceModelDownloader: Simple go utility to
download HuggingFace Models and Datasets

$ aqua g -i bodaay/HuggingFaceModelDownloader

How to confirm if this package works well

Reviewers aren't necessarily familiar with this package, so please describe how to confirm if this package works well.
Please confirm if this package works well yourself as much as possible.

Command and output

$ hfdownloader --help
a Simple HuggingFace Models Downloader Utility
Version: 1.2.9
Running on: /root/aquaproj-aqua/pkgs/github_release/github.com/bodaay/HuggingFaceModelDownloader/1.2.9/hfdownloader_linux_arm64_1.2.9/hfdownloader_linux_arm64_1.2.9

Usage:
  hfdowloader [flags]

Flags:
  -m, --model string         Model/Dataset name (required if dataset not set)
                             You can supply filters for required LFS model files
                             ex:  ModelName:q4_0,q8_1
                             ex:  TheBloke/WizardLM-Uncensored-Falcon-7B-GGML:fp16
  -d, --dataset string       Model/Dataset name (required if model not set)
  -b, --branch string        ModModel/Datasetel branch (optional) (default "main")
  -s, --storage string       Storage path (optional) (default "Storage")
  -k, --skipSHA              Skip SHA256 Hash Check, sometimes you just need to download missing files without wasting time waiting (optional)
  -f, --appendFilterFolder   This will append the filter name to the folder, use it for GGML qunatizatized filterd download only (optional)
  -c, --concurrent int       Number of LFS concurrent connections (optional) (default 5)
  -t, --token string         HuggingFace Access Token, this can be automatically supplied by env variable 'HUGGING_FACE_HUB_TOKEN' or .env file, required for some Models/Datasets, you still need to manually accept agreement if model requires it (optional)
  -i, --install              Install the binary to the OS default bin folder, Unix-like operating systems only
  -p, --installPath string   install Path (optional) (default "/usr/local/bin/")
      --maxRetries int       Max number of retries (optional) (default 3)
      --retryInterval int    Retry interval in seconds (optional) (default 5)
  -h, --help                 help for hfdowloader

[bodaay/HuggingFaceModelDownloader](https://github.com/bodaay/HuggingFaceModelDownloader): Simple go utility to download HuggingFace Models and Datasets
@suzuki-shunsuke suzuki-shunsuke added the enhancement New feature or request label Feb 8, 2024
@suzuki-shunsuke
Copy link
Member

Thank you!

@suzuki-shunsuke suzuki-shunsuke added this to the v4.129.0 milestone Feb 8, 2024
@suzuki-shunsuke suzuki-shunsuke merged commit 1c1d857 into aquaproj:main Feb 8, 2024
16 checks passed
@suzuki-shunsuke
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants