Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http upload endpoint to pack many files into a car? #2295

Open
tomjohnhall opened this issue Jul 22, 2023 · 1 comment
Open

http upload endpoint to pack many files into a car? #2295

tomjohnhall opened this issue Jul 22, 2023 · 1 comment
Labels
kind/enhancement A net-new feature or improvement to an existing feature need/triage Needs initial labeling and prioritization

Comments

@tomjohnhall
Copy link

I've been working on an integration which uses the upload endpoint, mostly so that I can work with streams rather than buffer files in memory - they generally aren't large files but it would be more efficient to just pipe the data straight to the endpoint. I was hoping to be able to upload multiple files and get back a car just like the putFiles method of the js SDK. Digging into the code I noticed this comment:

packages/api/src/upload.js

// TODO: do we want to wrap file uploads like we do car uploads from the client?
  // this path used to send the files to cluster and we didn't wrap, so we dont here for consistency with the old ways.
  const { car } = await packToBlob({
    input,
    maxChunkSize: MAX_CHUNK_SIZE,
    maxChildrenPerNode: MAX_CHILDREN_PER_NODE,
    wrapWithDirectory: false
  })

How feasible would it be to have a version of the upload endpoint which does take a number of files and return a car which contains them? At the moment I'm getting back a carCid which doesn't resolve on the gateway, and a cid which returns the first file, and my account shows that single file uploaded.

That said, maybe my request is formatted slightly wrong for multiple files - seems fine but perhaps I'm missing something!

Thanks for any help!

@tomjohnhall tomjohnhall added kind/enhancement A net-new feature or improvement to an existing feature need/triage Needs initial labeling and prioritization labels Jul 22, 2023
@alanshaw
Copy link
Member

The put method of the client takes a files parameter which is an array of Filelike. Filelike are just objects with a stream() function that returns a stream.

With this you should be able to upload multiple files, and stream them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement A net-new feature or improvement to an existing feature need/triage Needs initial labeling and prioritization
Projects
None yet
Development

No branches or pull requests

2 participants