Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to peek at the next decoder character #18

Closed
brandonf2002 opened this issue Mar 3, 2024 · 1 comment
Closed

Add ability to peek at the next decoder character #18

brandonf2002 opened this issue Mar 3, 2024 · 1 comment

Comments

@brandonf2002
Copy link

Hi, I'm currently writing a parser/lexer for a language that makes use of UTF-8 characters as part of its grammar and thought it would be useful to be able to "peek" at the next character in the decoder stream without advancing the decoder position forward.

There are plenty of work arounds for this situation but I thought it might be a useful feature to have built in.

I am proposing a function that would look like:

val decoder_peek : decoder ->
  [ `Await | `Uchar of Uchar.t | `End | `Malformed of string]

I would be more than happy to write this PR myself, I am just making this issue to see if a patch for this would be accepted.

@dbuenzli
Copy link
Owner

dbuenzli commented Mar 3, 2024

Thanks for making an issue first. The answer is no. For the following reasons:

  1. It's not too hard to integrate that lookahead in your lexer or parser state. See here for an example.

  2. In general uutf is in maintenance mode, I don't plan to add any new feature and I encourage people to directly use the stdlib decoders that have been integrated in OCaml 4.14.0 (Add UTF codec and validations support to the Stdlib ocaml/ocaml#10710). They are also likely more performant (at least they allocate less on decodes).

@dbuenzli dbuenzli closed this as completed Mar 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants