Navigation Menu

Skip to content

Commit

Permalink
docs: Clarified how clients should work with resumable uploads (#1457)
Browse files Browse the repository at this point in the history 
* docs: Clarified how clients should work with resumable uploads
docs: Clarified ListNotifications pagination
docs: Made "live generation" wording consistent with docs for other Cloud Storage APIs
fix: Made negative offsets larger in magnitude that object size return the full object for ReadObject operations
fix: Made naming format for Logging.log_bucket be a path rather than raw bucket name, to be consistent with the rest of the API
feat: Changed Custom Dual Regions to be specified in a proto message rather than a syntactic encoding within the bucket location
feat: Added etag support

PiperOrigin-RevId: 455465509

Source-Link: googleapis/googleapis@0470cd6

Source-Link: googleapis/googleapis-gen@1b16673
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMWIxNjY3Mzk1ZTcwN2E2MGFiY2EwOTNlOGZmZDUxMmY0YTBhMDExNiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* chore: update clirr-ignored-differences.xml

Allow list new etag and CustomPlacementConfig properties

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: BenWhitehead <BenWhitehead@users.noreply.github.com>
  • Loading branch information
@gcf-owl-bot @BenWhitehead
3 people committed Jun 17, 2022
1 parent fef5059 commit 15baa27
Show file tree
Hide file tree
Showing 32 changed files with 3,281 additions and 601 deletions.
36 changes: 28 additions & 8 deletions gapic-google-cloud-storage-v2/src/main/java/com/google/storage/v2/StorageClient.java
View file
Expand Up @@ -2035,8 +2035,8 @@ public final ServerStreamingCallable<ReadObjectRequest, ReadObjectResponse> read
*
* @param object The object to update. The object's bucket and name fields are used to identify
* the object to update. If present, the object's generation field selects a specific revision
* of this object whose metadata should be updated. Otherwise, assumes the current, live
* version of the object.
* of this object whose metadata should be updated. Otherwise, assumes the live version of the
* object.
* @param updateMask List of fields to be updated.
* <p>To specify ALL fields, equivalent to the JSON API's "update" function, specify a single
* field with the value `&#42;`. Note: not recommended. If a new field is introduced at a
Expand Down Expand Up @@ -2125,19 +2125,39 @@ public final UnaryCallable<UpdateObjectRequest, Object> updateObjectCallable() {
* preconditions. Additionally, the final message must set 'finish_write' to true, or else it is
* an error.
*
* <p>For a resumable write, the client should instead call `StartResumableWrite()` and provide
* that method an `WriteObjectSpec.` They should then attach the returned `upload_id` to the first
* message of each following call to `Create`. If there is an error or the connection is broken
* during the resumable `Create()`, the client should check the status of the `Create()` by
* calling `QueryWriteStatus()` and continue writing from the returned `persisted_size`. This may
* be less than the amount of data the client previously sent.
* <p>For a resumable write, the client should instead call `StartResumableWrite()`, populating a
* `WriteObjectSpec` into that request. They should then attach the returned `upload_id` to the
* first message of each following call to `WriteObject`. If the stream is closed before finishing
* the upload (either explicitly by the client or due to a network error or an error response from
* the server), the client should do as follows: - Check the result Status of the stream, to
* determine if writing can be resumed on this stream or must be restarted from scratch (by
* calling `StartResumableWrite()`). The resumable errors are DEADLINE_EXCEEDED, INTERNAL, and
* UNAVAILABLE. For each case, the client should use binary exponential backoff before retrying.
* Additionally, writes can be resumed after RESOURCE_EXHAUSTED errors, but only after taking
* appropriate measures, which may include reducing aggregate send rate across clients and/or
* requesting a quota increase for your project. - If the call to `WriteObject` returns `ABORTED`,
* that indicates concurrent attempts to update the resumable write, caused either by multiple
* racing clients or by a single client where the previous request was timed out on the client
* side but nonetheless reached the server. In this case the client should take steps to prevent
* further concurrent writes (e.g., increase the timeouts, stop using more than one process to
* perform the upload, etc.), and then should follow the steps below for resuming the upload. -
* For resumable errors, the client should call `QueryWriteStatus()` and then continue writing
* from the returned `persisted_size`. This may be less than the amount of data the client
* previously sent. Note also that it is acceptable to send data starting at an offset earlier
* than the returned `persisted_size`; in this case, the service will skip data at offsets that
* were already persisted (without checking that it matches the previously written data), and
* write only the data starting from the persisted offset. This behavior can make client-side
* handling simpler in some cases.
*
* <p>The service will not view the object as complete until the client has sent a
* `WriteObjectRequest` with `finish_write` set to `true`. Sending any requests on a stream after
* sending a request with `finish_write` set to `true` will cause an error. The client
* &#42;&#42;should&#42;&#42; check the response it receives to determine how much data the
* service was able to commit and whether the service views the object as complete.
*
* <p>Attempting to resume an already finalized object will result in an OK status, with a
* WriteObjectResponse containing the finalized object's metadata.
*
* <p>Sample code:
*
* <pre>{@code
Expand Down
25 changes: 25 additions & 0 deletions gapic-google-cloud-storage-v2/src/test/java/com/google/storage/v2/StorageClientTest.java
View file
Expand Up @@ -175,6 +175,7 @@ public void getBucketTest() throws Exception {
Bucket.newBuilder()
.setName(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setBucketId("bucketId-1603305307")
.setEtag("etag3123477")
.setProject(ProjectName.of("[PROJECT]").toString())
.setMetageneration(1048558813)
.setLocation("location1901043637")
Expand All @@ -198,6 +199,7 @@ public void getBucketTest() throws Exception {
.setRetentionPolicy(Bucket.RetentionPolicy.newBuilder().build())
.setIamConfig(Bucket.IamConfig.newBuilder().build())
.setSatisfiesPzs(true)
.setCustomPlacementConfig(Bucket.CustomPlacementConfig.newBuilder().build())
.setAutoclass(Bucket.Autoclass.newBuilder().build())
.build();
mockStorage.addResponse(expectedResponse);
Expand Down Expand Up @@ -238,6 +240,7 @@ public void getBucketTest2() throws Exception {
Bucket.newBuilder()
.setName(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setBucketId("bucketId-1603305307")
.setEtag("etag3123477")
.setProject(ProjectName.of("[PROJECT]").toString())
.setMetageneration(1048558813)
.setLocation("location1901043637")
Expand All @@ -261,6 +264,7 @@ public void getBucketTest2() throws Exception {
.setRetentionPolicy(Bucket.RetentionPolicy.newBuilder().build())
.setIamConfig(Bucket.IamConfig.newBuilder().build())
.setSatisfiesPzs(true)
.setCustomPlacementConfig(Bucket.CustomPlacementConfig.newBuilder().build())
.setAutoclass(Bucket.Autoclass.newBuilder().build())
.build();
mockStorage.addResponse(expectedResponse);
Expand Down Expand Up @@ -301,6 +305,7 @@ public void createBucketTest() throws Exception {
Bucket.newBuilder()
.setName(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setBucketId("bucketId-1603305307")
.setEtag("etag3123477")
.setProject(ProjectName.of("[PROJECT]").toString())
.setMetageneration(1048558813)
.setLocation("location1901043637")
Expand All @@ -324,6 +329,7 @@ public void createBucketTest() throws Exception {
.setRetentionPolicy(Bucket.RetentionPolicy.newBuilder().build())
.setIamConfig(Bucket.IamConfig.newBuilder().build())
.setSatisfiesPzs(true)
.setCustomPlacementConfig(Bucket.CustomPlacementConfig.newBuilder().build())
.setAutoclass(Bucket.Autoclass.newBuilder().build())
.build();
mockStorage.addResponse(expectedResponse);
Expand Down Expand Up @@ -370,6 +376,7 @@ public void createBucketTest2() throws Exception {
Bucket.newBuilder()
.setName(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setBucketId("bucketId-1603305307")
.setEtag("etag3123477")
.setProject(ProjectName.of("[PROJECT]").toString())
.setMetageneration(1048558813)
.setLocation("location1901043637")
Expand All @@ -393,6 +400,7 @@ public void createBucketTest2() throws Exception {
.setRetentionPolicy(Bucket.RetentionPolicy.newBuilder().build())
.setIamConfig(Bucket.IamConfig.newBuilder().build())
.setSatisfiesPzs(true)
.setCustomPlacementConfig(Bucket.CustomPlacementConfig.newBuilder().build())
.setAutoclass(Bucket.Autoclass.newBuilder().build())
.build();
mockStorage.addResponse(expectedResponse);
Expand Down Expand Up @@ -527,6 +535,7 @@ public void lockBucketRetentionPolicyTest() throws Exception {
Bucket.newBuilder()
.setName(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setBucketId("bucketId-1603305307")
.setEtag("etag3123477")
.setProject(ProjectName.of("[PROJECT]").toString())
.setMetageneration(1048558813)
.setLocation("location1901043637")
Expand All @@ -550,6 +559,7 @@ public void lockBucketRetentionPolicyTest() throws Exception {
.setRetentionPolicy(Bucket.RetentionPolicy.newBuilder().build())
.setIamConfig(Bucket.IamConfig.newBuilder().build())
.setSatisfiesPzs(true)
.setCustomPlacementConfig(Bucket.CustomPlacementConfig.newBuilder().build())
.setAutoclass(Bucket.Autoclass.newBuilder().build())
.build();
mockStorage.addResponse(expectedResponse);
Expand Down Expand Up @@ -591,6 +601,7 @@ public void lockBucketRetentionPolicyTest2() throws Exception {
Bucket.newBuilder()
.setName(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setBucketId("bucketId-1603305307")
.setEtag("etag3123477")
.setProject(ProjectName.of("[PROJECT]").toString())
.setMetageneration(1048558813)
.setLocation("location1901043637")
Expand All @@ -614,6 +625,7 @@ public void lockBucketRetentionPolicyTest2() throws Exception {
.setRetentionPolicy(Bucket.RetentionPolicy.newBuilder().build())
.setIamConfig(Bucket.IamConfig.newBuilder().build())
.setSatisfiesPzs(true)
.setCustomPlacementConfig(Bucket.CustomPlacementConfig.newBuilder().build())
.setAutoclass(Bucket.Autoclass.newBuilder().build())
.build();
mockStorage.addResponse(expectedResponse);
Expand Down Expand Up @@ -909,6 +921,7 @@ public void updateBucketTest() throws Exception {
Bucket.newBuilder()
.setName(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setBucketId("bucketId-1603305307")
.setEtag("etag3123477")
.setProject(ProjectName.of("[PROJECT]").toString())
.setMetageneration(1048558813)
.setLocation("location1901043637")
Expand All @@ -932,6 +945,7 @@ public void updateBucketTest() throws Exception {
.setRetentionPolicy(Bucket.RetentionPolicy.newBuilder().build())
.setIamConfig(Bucket.IamConfig.newBuilder().build())
.setSatisfiesPzs(true)
.setCustomPlacementConfig(Bucket.CustomPlacementConfig.newBuilder().build())
.setAutoclass(Bucket.Autoclass.newBuilder().build())
.build();
mockStorage.addResponse(expectedResponse);
Expand Down Expand Up @@ -1043,6 +1057,7 @@ public void getNotificationTest() throws Exception {
Notification.newBuilder()
.setName(NotificationName.of("[PROJECT]", "[BUCKET]", "[NOTIFICATION]").toString())
.setTopic("topic110546223")
.setEtag("etag3123477")
.addAllEventTypes(new ArrayList<String>())
.putAllCustomAttributes(new HashMap<String, String>())
.setObjectNamePrefix("objectNamePrefix-1978236516")
Expand Down Expand Up @@ -1086,6 +1101,7 @@ public void getNotificationTest2() throws Exception {
Notification.newBuilder()
.setName(NotificationName.of("[PROJECT]", "[BUCKET]", "[NOTIFICATION]").toString())
.setTopic("topic110546223")
.setEtag("etag3123477")
.addAllEventTypes(new ArrayList<String>())
.putAllCustomAttributes(new HashMap<String, String>())
.setObjectNamePrefix("objectNamePrefix-1978236516")
Expand Down Expand Up @@ -1129,6 +1145,7 @@ public void createNotificationTest() throws Exception {
Notification.newBuilder()
.setName(NotificationName.of("[PROJECT]", "[BUCKET]", "[NOTIFICATION]").toString())
.setTopic("topic110546223")
.setEtag("etag3123477")
.addAllEventTypes(new ArrayList<String>())
.putAllCustomAttributes(new HashMap<String, String>())
.setObjectNamePrefix("objectNamePrefix-1978236516")
Expand Down Expand Up @@ -1175,6 +1192,7 @@ public void createNotificationTest2() throws Exception {
Notification.newBuilder()
.setName(NotificationName.of("[PROJECT]", "[BUCKET]", "[NOTIFICATION]").toString())
.setTopic("topic110546223")
.setEtag("etag3123477")
.addAllEventTypes(new ArrayList<String>())
.putAllCustomAttributes(new HashMap<String, String>())
.setObjectNamePrefix("objectNamePrefix-1978236516")
Expand Down Expand Up @@ -1309,6 +1327,7 @@ public void composeObjectTest() throws Exception {
Object.newBuilder()
.setName("name3373707")
.setBucket(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setEtag("etag3123477")
.setGeneration(305703192)
.setMetageneration(1048558813)
.setStorageClass("storageClass871353277")
Expand Down Expand Up @@ -1482,6 +1501,7 @@ public void getObjectTest() throws Exception {
Object.newBuilder()
.setName("name3373707")
.setBucket(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setEtag("etag3123477")
.setGeneration(305703192)
.setMetageneration(1048558813)
.setStorageClass("storageClass871353277")
Expand Down Expand Up @@ -1550,6 +1570,7 @@ public void getObjectTest2() throws Exception {
Object.newBuilder()
.setName("name3373707")
.setBucket(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setEtag("etag3123477")
.setGeneration(305703192)
.setMetageneration(1048558813)
.setStorageClass("storageClass871353277")
Expand Down Expand Up @@ -1692,6 +1713,7 @@ public void updateObjectTest() throws Exception {
Object.newBuilder()
.setName("name3373707")
.setBucket(BucketName.of("[PROJECT]", "[BUCKET]").toString())
.setEtag("etag3123477")
.setGeneration(305703192)
.setMetageneration(1048558813)
.setStorageClass("storageClass871353277")
Expand Down Expand Up @@ -2352,6 +2374,7 @@ public void getHmacKeyTest() throws Exception {
.setState("state109757585")
.setCreateTime(Timestamp.newBuilder().build())
.setUpdateTime(Timestamp.newBuilder().build())
.setEtag("etag3123477")
.build();
mockStorage.addResponse(expectedResponse);

Expand Down Expand Up @@ -2399,6 +2422,7 @@ public void getHmacKeyTest2() throws Exception {
.setState("state109757585")
.setCreateTime(Timestamp.newBuilder().build())
.setUpdateTime(Timestamp.newBuilder().build())
.setEtag("etag3123477")
.build();
mockStorage.addResponse(expectedResponse);

Expand Down Expand Up @@ -2534,6 +2558,7 @@ public void updateHmacKeyTest() throws Exception {
.setState("state109757585")
.setCreateTime(Timestamp.newBuilder().build())
.setUpdateTime(Timestamp.newBuilder().build())
.setEtag("etag3123477")
.build();
mockStorage.addResponse(expectedResponse);

Expand Down

0 comments on commit 15baa27

Please sign in to comment.