hfl

hfl is a single-file static configuration pipeline tool for HF.

hfl only works with common NLP tasks & is configured with PyTorch.

This is currently experimental (WIP).

Motivation

This section is incomplete.

Feature Matrix

• Basic pipelines.
• Inherited checkpoints (model & tokenizer).
• Multi-sequence tokenization.
• Training on train datasets (local & AWS SageMaker).
• Deploying fine-tuned models (push_to_hub).
• Localized metrics from test sets.
• Specifying custom preprocessor scripts.
• Integration with custom training loops.

Installation

Target a global pip installation to this repository.

Build from Source

Windows Compat

You can either make create_env_win_cmd or if you require to change the execution policy then make create_env_win_pow followed by the relevant load target.

Usage

This section is incomplete.