Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch: Support HTTP Range Requests #53

Open
dr-vortex opened this issue Oct 25, 2023 · 0 comments
Open

Fetch: Support HTTP Range Requests #53

dr-vortex opened this issue Oct 25, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@dr-vortex
Copy link
Member

dr-vortex commented Oct 25, 2023

For large files (100MB+), downloading and storing the entire file in-memory is prohibitively expensive, and may not > be needed if only small parts of the file are needed.

It would be great to support HTTP range requests in both synchronous and asynchronous modes.

Some important questions:

  • How large should the chunks be? Should that be configurable?
  • Should we prefetch chunks? What prefetch policies are useful?
  • What configuration knobs should we expose to the user?
  • What eviction policy should we use? FIFO? How much data should we keep in-memory?
  • Do browsers cache data retrieved via HTTP range requests? This should make re-acquiring previously evicted blocks much cheaper / avoid draining the server over a long session.

I don't anticipate being able to work on this anytime soon, but I'm recording my thoughts here for now.

~ @jvilk

jvilk/BrowserFS#219

@dr-vortex dr-vortex added the enhancement New feature or request label Oct 25, 2023
@dr-vortex dr-vortex changed the title DownloadFS: Support HTTP Range Requests Support HTTP Range Requests Mar 20, 2024
@dr-vortex dr-vortex transferred this issue from zen-fs/core Mar 23, 2024
@dr-vortex dr-vortex transferred this issue from zen-fs/fetch May 8, 2024
@dr-vortex dr-vortex changed the title Support HTTP Range Requests Fetch: Support HTTP Range Requests May 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant