Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add upload functionality #214

Merged
merged 3 commits into from Apr 1, 2020
Merged

Conversation

dmitry-fa
Copy link
Contributor

Fixes #179

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Mar 30, 2020
@codecov
Copy link

codecov bot commented Mar 30, 2020

Codecov Report

Merging #214 into master will increase coverage by 0.17%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #214      +/-   ##
============================================
+ Coverage     63.50%   63.67%   +0.17%     
- Complexity      540      548       +8     
============================================
  Files            30       31       +1     
  Lines          4759     4782      +23     
  Branches        427      428       +1     
============================================
+ Hits           3022     3045      +23     
  Misses         1577     1577              
  Partials        160      160              
Impacted Files Coverage Δ Complexity Δ
...va/com/google/cloud/storage/StorageOperations.java 100.00% <100.00%> (ø) 8.00 <8.00> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 789ceaa...dec00e8. Read the comment docs.

* }
* }</pre>
*/
public final class StorageUtils {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Utils" classes usually don't have object state or non-static methods. Perhaps there's a better name here? Uploader or StorageUploader perhaps?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class could be used not only of uploading, it could contain downloading or another functionality as well. Would it be okay to rename it to StorageHelper?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works. Another possibility: StorageTransmitter.

However, also consider whether the Storage field works better as a method argument.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a very rare case when someone has more than one instance of the storage. So, passing this instance to every method doesn't seem to be convenient. Methods with 4 arguments doesn't look as easy to understand.
Having a class above the storage will certainly give some freedom in extending API without breaking compatibility. To me 'Transmitter' sounds like something closed to the physical layer. 'Helper' is very generic. May be StorageOperations or StorageFunctions fits better?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StorageMover? StorageTransporter?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like StorageOperations, actually.

@dmitry-fa dmitry-fa requested a review from elharo March 31, 2020 16:50
/**
* Utility methods to perform various operations with the Storage such as upload.
*
* <p>Example of uploading files from a folder:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"directory" might or might not be clearer here. Depending on whether the reader is coming from Mac, Windows, or Linux. However Java does call this directory, so we should probably stick to that.

* StorageUtils utils = StorageUtils.create(storage);
* for (File file: folder.listFiles()) {
* if (!file.isDirectory()) {
* BlobInfo blobInfo = BlobInfo.newBuilder(BUCKET, file.getName()).build();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two space indents in example

* }
* }</pre>
*/
public final class StorageUtils {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StorageMover? StorageTransporter?

public final class StorageUtils {

/** The instance of the Storage the utilities are associated with. */
public final Storage storage;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

important: private

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StorageMover? StorageTransporter?

This class can be used not only for upload/download operation, but for any operation with the storage. That will allow to add new functionality without extending the Storage interface and breaking API.

May be StorageOperations or StorageFunctions feet better?

* @param storage the Storage
* @return an instance which refers to {@code storage}
*/
public static StorageUtils create(Storage storage) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no better than a constructor. Remove it.


/**
* Uploads the given {@code path} to the blob using {@link Storage#writer} and the given {@code
* bufferSize}. By default any MD5 and CRC32C values in the given {@code blobInfo} are ignored
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete "the given"

* unless requested via the {@link Storage.BlobWriteOption#md5Match()} and {@link
* Storage.BlobWriteOption#crc32cMatch()} options. Folder upload is not supported.
*
* <p>{@link #upload(BlobInfo, Path, Storage.BlobWriteOption...)} invokes this one with a buffer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is "this one"?

}

/**
* Uploads the given {@code content} to the blob using {@link Storage#writer}. By default any MD5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Uploads the given {@code content}" --> Reads bytes from an input stream and uploads those bytes to the blob

*
* <pre>{@code
* BlobId blobId = BlobId.of(bucketName, blobName);
* byte[] content = "Hello, world".getBytes(UTF_8);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StandardCharsets.UTF_8

* <pre>{@code
* File folder = new File("pictures/");
* StorageUtils utils = StorageUtils.create(storage);
* for (File file: folder.listFiles()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why you have this example at all. It doesn't handle subdirectories.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example demonstrates that:

  • StorageUtils instance can be reused
  • Upload of folders is not supported

To upload a folder with subfolders one should case about nuances like correct handling of symbolic links. Getting the blob name for files in subfolders requires an extra code. So the example will be too complicated.
I can remove this one.

@dmitry-fa
Copy link
Contributor Author

@elharo thanks a lot for your comments

@dmitry-fa dmitry-fa merged commit 7beb99d into googleapis:master Apr 1, 2020
@frankyn frankyn mentioned this pull request Apr 1, 2020
frankyn added a commit that referenced this pull request Apr 2, 2020
frankyn added a commit that referenced this pull request Apr 2, 2020
@release-please release-please bot mentioned this pull request Apr 2, 2020
@dmitry-fa dmitry-fa deleted the storageUtils branch June 26, 2020 07:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add Storage.upload(Path)
3 participants