Add Blob.uploadFrom(InputStream)

[broady]

We have Blob.downloadTo(OutputStream) (and other downloadFrom variants)

Should probably have uploadFrom with a stream. People currently have to buffer/chunk their writes manually.

/cc @crwilcox

[dmitry-fa]

New methods of Blob are suggested:

public void uploadFrom(Path path, BlobWriteOption... options) {...}
public void uploadFrom(Path path, int chunkSize, BlobWriteOption... options) {...}
public void uploadFrom(InputStream input, BlobWriteOption... options) {...}

  /**
   * Uploads the given content to this blob using specified blob write options and the provided
   * chunk size. The default chunk size is 2 MB. Larger chunk sizes may improve the upload
   * performance but require more memory. It could cause OutOfMemoryError or add significant garbage
   * collection overhead. Chunk sizes which are less than 256 KB are not allowed, they will be
   * treated as 256 KB.
   *
   * @param input content to be uploaded
   * @param chunkSize the minimum size that will be written by a single RPC.
   * @param options blob write options
   * @throws StorageException upon failure
   */
  public void uploadFrom(InputStream input, int chunkSize, BlobWriteOption... options) {
    chunkSize = Math.max(chunkSize, 262144); // adjust with MinChunkSize of BaseWriteChannel
    try (WriteChannel writer = storage.writer(this, options)) {
      writer.setChunkSize(chunkSize);
      byte[] buffer = new byte[chunkSize];
      int length;
      while ((length = input.read(buffer)) >= 0) {
        writer.write(ByteBuffer.wrap(buffer, 0, length));
      }
    } catch (IOException e) {
      throw new StorageException(e);
    }
  }

[dmitry-fa]

The following extension to the Blob class is proposed:

  public Blob uploadFrom(Path path, BlobWriteOption... options)
  public Blob uploadFrom(InputStream input, BlobWriteOption... options)

  /**
   * Uploads the given content to the storage using specified write channel and the given
   * buffer size. The default buffer size is 15 MB. Larger buffer sizes may improve the upload
   * performance but require more memory. It could cause OutOfMemoryError or add significant garbage
   * collection overhead. Buffer sizes which are less than 256 KB are not allowed, they will be
   * treated as 256 KB.
   *
   * <p>Note that this method does not close neither InputStream nor WriterChannel</p>
   *
   * <p>Example of uploading:
   *
   * <pre>{@code
   * Path file = Paths.get("humongous.file");
   * try (InputStream input = Files.newInputStream(file); WriteChannel writer = storage.writer(blobInfo)) {
   *     Blob.upload(input, writer, 150 * 1024 * 1024);
   * }
   *
   * @param input content to be uploaded
   * @param writer channel
   * @param bufferSize size of the buffer to read from input and send over writer
   */
  public static void upload(InputStream input, WriteChannel writer, int bufferSize)

[dmitry-fa]

Similar extension to the Storage interface is planned to be proposed with a separate PR

[dmitry-fa]

fix was reverted by #190

[dmitry-fa]

#214 fixes this issue as well