Is it possible to fine-tune without Azure AI studio?

[mark86v1]

Is it possible to implement RAFT without Azure AI studio? I am planning to use open-source models.

[Danielskry]

Yes, you can fine-tune or implement RAFT without Azure AI Studio, provided you have access to the appropriate hardware. One challenging aspect is determining whether the model you wish to fine-tune fits within the available vRAM memory (e.g., loading a model onto CUDA). I suggest using a calculator (can be found on Azure, Hugging Face, etc.) to assist with this process of selecting the proper model and GPU. Fine-tuning models can quickly become expensive and may exceed the capabilities of standard consumer GPUs, which is why many opt for platforms like Azure AI Studio for these tasks.

For example, Tianjun and Shishir used "Llama 2 7B is also a perfect model for training on 4 A100-40G GPUs and serving on a single GPU" (given Microsoft blog post) to implement RAFT on a Llama 2 7B model. The price for one A100 40GB GPU is now roughly $8,200 USD.

[songole]

is there a skypilot recipe to finetune this on any cloud?