Added grammar (and json schemas and CPU-only Dockerfile) support (from ollama/ollama PR #1606)

[markcda]

Updated version of #1606.

[qkxie]

waiting for merging

[mitar]

This looks really cool. Could this be merged? It is a simple PR which just passes options to llama.cpp.

[mitar]

Also, it would be useful to have llama.cpp which includes ggerganov/llama.cpp#6811 and ggerganov/llama.cpp#7424 because they bring performance improvements to grammar sampling.

[markcda]

Also, it would be useful to have llama.cpp which includes ggerganov/llama.cpp#6811 and ggerganov/llama.cpp#7424 because they bring performance improvements to grammar sampling.

I think I can merge it to my fork for now while this pull request wasn't merged.

[mitar]

@jmorganca There are bunch of PRs adding grammar support to Ollama, I think it would be great if this PR or some other could be merged, especially given that llama.cpp now supports also providing JSON schema directly. This is very powerful feature.

[mitar]

@markcda This PR is pretty ugly because you made PR from the main branch of your fork, not a feature branch. Also, you are merging upstream (this repository) changes into your branch, instead of rebasing. So there is bunch of useless commits from the view of this repository.

[mitar]

Also why is CPU only Dockerfile part of this PR and not another PR?

[coder543]

@mitar are you one of the maintainers here? These things would only matter if the maintainers were reviewing this PR... so far, they've shown no inclination to accept any of the grammar PRs.

[mitar]

No, I am not. But if PR hopes to be merged at all, then this matters. In general it is a good etiquette. Otherwise what's the point of the MR?

[coder543]

We don’t know what the maintainers care about.