Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delimit streams into records -- e.g. read --netstr #1905

Open
glyh opened this issue Apr 6, 2024 · 3 comments
Open

Delimit streams into records -- e.g. read --netstr #1905

glyh opened this issue Apr 6, 2024 · 3 comments

Comments

@glyh
Copy link
Collaborator

glyh commented Apr 6, 2024

For now json read parse the whole input, but it would be nice that the parser stops when it receive everything it need.

This is useful for implementing something like jsonrpc.

@glyh glyh added the feature label Apr 6, 2024
@andychu
Copy link
Contributor

andychu commented Apr 6, 2024

I think we are going to use length-delimited netstrings for most things

The headless shell already uses that

https://www.oilshell.org/release/0.21.0/doc/headless.html

https://github.com/oilshell/oil/blob/master/client/py_fanos.py

So you can put JSON in netstrings like:

2:{},
4:[42],

I think it should be read --netstring or something

(Patches welcome! The Python code in FANOS may be copied / adapted)


If people really want JSON-RPC, it should be wrapped in something like curl

e.g. we don't implement HTTP in ysh directly, but we shell out to curl, or we shell out to a Go binary

Same with JSON-RPC

@andychu
Copy link
Contributor

andychu commented Apr 6, 2024

I wrote this doc a long time ago - https://www.oilshell.org/release/0.21.0/doc/framing.html

Basically I think we separate

  • delimiting records in a stream
  • parsing within those records

@glyh
Copy link
Collaborator Author

glyh commented Apr 6, 2024

fanos.c is gone.

@andychu andychu changed the title [FR] json read on a stream pipeline Delimit streams into records -- e.g. read --netstr Apr 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants