Possible to support llama.c/GGUF/GGML

[TomExMachina]

I'm wondering if GGUF/GGML can be supported by skypilot. Thank you

[Michaelvll]

It should be able to support those. Would you like to submit a PR for it?

[TomExMachina]

I meant llama.cpp in the title. I'll certainly give it a shot once I'm more familiar with the library. I'm just getting started by deploying various models. At the moment I'm trying to figure out why I get nonsense generations from WizardLM/WizardCoder-Python-34B-V1.0 vllm/skypilot but that's for another post.

I'll probably go for the lower hanging fruit of runpod support before llama.cpp.