Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Constrained sampling based on EBNF grammars #354

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

seanmor5
Copy link
Contributor

Matches the LlamaCPP behavior. I finished the EBNF parser which encodes the grammar in the same way as the implementation from: huggingface/transformers#27557

Unfortunately I think we may have to refactor or redesign much of the logic for processing acceptances in a way that is compatible with XLA. I know we can mimic a stack like data-structure (which I started to do), and I believe we can mimic the trie as well as containers. The issue I'm having is how possible it is to implement something like https://github.com/huggingface/transformers/pull/27557/files#diff-b7135bf8eda80faf271e4c9588eae893ebad019d2508df2f0afbe5b7ad5bbf4eR389 in a non-recursive way. Unless my understanding is incorrect and the actual incremental grammar acceptance process is the fixed depending on grammar and we can "compile" the acceptance up front

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant