Skip to content
This repository has been archived by the owner on Mar 25, 2024. It is now read-only.

Make Iterator implementation of Deserializer::from_reader lazy #390

Open
quilir opened this issue Sep 22, 2023 · 0 comments
Open

Make Iterator implementation of Deserializer::from_reader lazy #390

quilir opened this issue Sep 22, 2023 · 0 comments

Comments

@quilir
Copy link

quilir commented Sep 22, 2023

The current implementation of Iterator for Deserializer created by from_reader constructor is eager. Upon calling next() for the first time, it uses Loader to load all the contents via read_to_end() on reader.

Such implementation is unfortunate for two reasons:

  • it does not allow for online processing of documents on readers without EOF
  • it wastes space and time, e.g. if someone needs to load only a single or few first documents from a large reader

I suggest loading only a single document from the reader at the time when calling next(). It could replace the current implementation or be optional feature accessible only by, for example, from_reader_lazy() constructor

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant