Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add samples for object retention #2417

Merged
merged 13 commits into from
Feb 26, 2024
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,13 @@ implementation 'com.google.cloud:google-cloud-storage'
If you are using Gradle without BOM, add this to your dependencies:

```Groovy
implementation 'com.google.cloud:google-cloud-storage:2.33.0'
implementation 'com.google.cloud:google-cloud-storage:2.34.0'
```

If you are using SBT, add this to your dependencies:

```Scala
libraryDependencies += "com.google.cloud" % "google-cloud-storage" % "2.33.0"
libraryDependencies += "com.google.cloud" % "google-cloud-storage" % "2.34.0"
```
<!-- {x-version-update-end} -->

Expand Down Expand Up @@ -237,6 +237,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/java-storage/tree/
| Create Bucket | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/bucket/CreateBucket.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/bucket/CreateBucket.java) |
| Create Bucket Dual Region | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/bucket/CreateBucketDualRegion.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/bucket/CreateBucketDualRegion.java) |
| Create Bucket Pub Sub Notification | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/bucket/CreateBucketPubSubNotification.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/bucket/CreateBucketPubSubNotification.java) |
| Create Bucket With Object Retention | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/bucket/CreateBucketWithObjectRetention.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/bucket/CreateBucketWithObjectRetention.java) |
| Create Bucket With Storage Class And Location | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/bucket/CreateBucketWithStorageClassAndLocation.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/bucket/CreateBucketWithStorageClassAndLocation.java) |
| Create Bucket With Turbo Replication | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/bucket/CreateBucketWithTurboReplication.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/bucket/CreateBucketWithTurboReplication.java) |
| Delete Bucket | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/bucket/DeleteBucket.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/bucket/DeleteBucket.java) |
Expand Down Expand Up @@ -322,6 +323,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/java-storage/tree/
| Rotate Object Encryption Key | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/object/RotateObjectEncryptionKey.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/object/RotateObjectEncryptionKey.java) |
| Set Event Based Hold | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/object/SetEventBasedHold.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/object/SetEventBasedHold.java) |
| Set Object Metadata | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/object/SetObjectMetadata.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/object/SetObjectMetadata.java) |
| Set Object Retention Policy | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/object/SetObjectRetentionPolicy.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/object/SetObjectRetentionPolicy.java) |
| Set Temporary Hold | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/object/SetTemporaryHold.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/object/SetTemporaryHold.java) |
| Stream Object Download | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/object/StreamObjectDownload.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/object/StreamObjectDownload.java) |
| Stream Object Upload | [source code](https://github.com/googleapis/java-storage/blob/main/samples/snippets/src/main/java/com/example/storage/object/StreamObjectUpload.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-storage&page=editor&open_in_editor=samples/snippets/src/main/java/com/example/storage/object/StreamObjectUpload.java) |
Expand Down Expand Up @@ -428,7 +430,7 @@ Java is a registered trademark of Oracle and/or its affiliates.
[kokoro-badge-link-5]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-storage/java11.html
[stability-image]: https://img.shields.io/badge/stability-stable-green
[maven-version-image]: https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-storage.svg
[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-storage/2.33.0
[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-storage/2.34.0
[authentication]: https://github.com/googleapis/google-cloud-java#authentication
[auth-scopes]: https://developers.google.com/identity/protocols/oauth2/scopes
[predefined-iam-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/*
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.storage.bucket;

// [START storage_create_bucket_with_object_retention]

import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.BucketInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

public class CreateBucketWithObjectRetention {
public static void createBucketWithObjectRetention(String projectId, String bucketName) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this sample required in docs? Wondering if users could be pointed to `BucketTargetOption.enableObjectRetention(true) instead of writing this sample in 8 languages.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's meant to go here: https://cloud.google.com/storage/docs/using-object-lock#enable-retentions

We want to have a sample for every use case on this page

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sg, thanks for clarifying.

// The ID of your GCP project
// String projectId = "your-project-id";

// The ID to give your GCS bucket
// String bucketName = "your-unique-bucket-name";

Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();

Bucket bucket =
storage.create(
BucketInfo.of(bucketName), Storage.BucketTargetOption.enableObjectRetention(true));

System.out.println(
"Created bucket "
+ bucket.getName()
+ " with object retention enabled setting: "
+ bucket.getObjectRetention().getMode().toString());
}
}

// [END storage_create_bucket_with_object_retention]
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ public static void getBucketMetadata(String projectId, String bucketName) {
System.out.println("StorageClass: " + bucket.getStorageClass().name());
System.out.println("TimeCreated: " + bucket.getCreateTime());
System.out.println("VersioningEnabled: " + bucket.versioningEnabled());
System.out.println("ObjectRetention: " + bucket.getObjectRetention());
if (bucket.getLabels() != null) {
System.out.println("\n\n\nLabels:");
for (Map.Entry<String, String> label : bucket.getLabels().entrySet()) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ public static void getObjectMetadata(String projectId, String bucketName, String
System.out.println("StorageClass: " + blob.getStorageClass());
System.out.println("TimeCreated: " + new Date(blob.getCreateTime()));
System.out.println("Last Metadata Update: " + new Date(blob.getUpdateTime()));
System.out.println("Object Retention Policy: " + blob.getRetention());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetBucketMetadata and this sample are missing test changes; please add them.

Boolean temporaryHoldIsEnabled = (blob.getTemporaryHold() != null && blob.getTemporaryHold());
System.out.println("temporaryHold: " + (temporaryHoldIsEnabled ? "enabled" : "disabled"));
Boolean eventBasedHoldIsEnabled =
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
/*
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.storage.object;

// [START storage_set_object_retention_policy]

import static java.time.OffsetDateTime.now;

import com.google.cloud.storage.Blob;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo.Retention;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageException;
import com.google.cloud.storage.StorageOptions;

public class SetObjectRetentionPolicy {
public static void setObjectRetentionPolicy(
String projectId, String bucketName, String objectName) throws StorageException {
// The ID of your GCP project
// String projectId = "your-project-id";

// The ID of your GCS bucket that has object retention enabled
// String bucketName = "your-unique-bucket-name";

// The ID of your GCS object
// String objectName = "your-object-name";

Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();
BlobId blobId = BlobId.of(bucketName, objectName);
Blob blob = storage.get(blobId);
if (blob == null) {
System.out.println("The object " + objectName + " was not found in " + bucketName);
return;
}

Blob updated =
blob.toBuilder()
.setRetention(
Retention.newBuilder()
.setMode(Retention.Mode.UNLOCKED)
.setRetainUntilTime(now().plusDays(10))
.build())
.build()
.update();

System.out.println("Retention policy for object " + objectName + " was set to:");
System.out.println(updated.getRetention().toString());

// To modify an existing policy on an Unlocked object, pass in the override parameter
blob.toBuilder()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like two samples in one; were you planning on splitting it or keeping as-is?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Planning to keep as is. I think it's important that a user sees both cases, so keeping them in one sample makes sure that it can't be missed.

.setRetention(
updated.getRetention().toBuilder().setRetainUntilTime(now().plusDays(9)).build())
.build()
.update(Storage.BlobTargetOption.overrideUnlockedRetention(true));

System.out.println("Retention policy for object " + objectName + " was updated to:");
System.out.println(storage.get(blobId).getRetention().toString());
}
}

// [END storage_set_object_retention_policy]
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
import com.example.storage.bucket.ChangeDefaultStorageClass;
import com.example.storage.bucket.ConfigureBucketCors;
import com.example.storage.bucket.CreateBucket;
import com.example.storage.bucket.CreateBucketWithObjectRetention;
import com.example.storage.bucket.CreateBucketWithStorageClassAndLocation;
import com.example.storage.bucket.CreateBucketWithTurboReplication;
import com.example.storage.bucket.DeleteBucket;
Expand Down Expand Up @@ -662,10 +663,14 @@ public void testLockRetentionPolicy() {
String tempBucket = RemoteStorageHelper.generateBucketName();
Bucket bucket = storage.create(BucketInfo.of(tempBucket));
assertNotNull(bucket);
SetRetentionPolicy.setRetentionPolicy(PROJECT_ID, tempBucket, 5L);
assertEquals(5L, (long) storage.get(tempBucket).getRetentionPeriod());
LockRetentionPolicy.lockRetentionPolicy(PROJECT_ID, tempBucket);
assertTrue(storage.get(tempBucket).retentionPolicyIsLocked());
try {
SetRetentionPolicy.setRetentionPolicy(PROJECT_ID, tempBucket, 5L);
assertEquals(5L, (long) storage.get(tempBucket).getRetentionPeriod());
LockRetentionPolicy.lockRetentionPolicy(PROJECT_ID, tempBucket);
assertTrue(storage.get(tempBucket).retentionPolicyIsLocked());
} finally {
storage.delete(tempBucket);
}
}

@Test
Expand All @@ -682,4 +687,18 @@ public void testUniformBucketLevelAccess() {
DisableUniformBucketLevelAccess.disableUniformBucketLevelAccess(PROJECT_ID, BUCKET);
assertFalse(storage.get(BUCKET).getIamConfiguration().isUniformBucketLevelAccessEnabled());
}

@Test
public void testCreateBucketWithObjectRetention() {
String tempBucket = RemoteStorageHelper.generateBucketName();

try {
CreateBucketWithObjectRetention.createBucketWithObjectRetention(PROJECT_ID, tempBucket);
assertNotNull(storage.get(tempBucket).getObjectRetention());
String snippetOutput = stdOutCaptureRule.getCapturedOutputAsUtf8String();
assertTrue(snippetOutput.contains("Enabled"));
} finally {
storage.delete(tempBucket);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
import com.example.storage.object.MoveObject;
import com.example.storage.object.RotateObjectEncryptionKey;
import com.example.storage.object.SetObjectMetadata;
import com.example.storage.object.SetObjectRetentionPolicy;
import com.example.storage.object.StreamObjectDownload;
import com.example.storage.object.StreamObjectUpload;
import com.example.storage.object.UploadEncryptedObject;
Expand Down Expand Up @@ -434,4 +435,28 @@ public void testBatchSetObjectMetadata() {
assertEquals("value", firstBlobMetadata.get("keyToAddOrUpdate"));
assertEquals("value", secondBlobMetadata.get("keyToAddOrUpdate"));
}

@Test
public void testSetObjectRetentionPolicy() {
String tempBucket = RemoteStorageHelper.generateBucketName();
storage.create(
BucketInfo.of(tempBucket), Storage.BucketTargetOption.enableObjectRetention(true));
String retentionBlob = "retentionblob";
storage.create(BlobInfo.newBuilder(tempBucket, retentionBlob).build());
assertNull(storage.get(tempBucket, retentionBlob).getRetention());
try {
SetObjectRetentionPolicy.setObjectRetentionPolicy(PROJECT_ID, tempBucket, retentionBlob);
assertNotNull(storage.get(tempBucket, retentionBlob).getRetention());
} finally {

storage
.get(tempBucket, retentionBlob)
.toBuilder()
.setRetention(null)
.build()
.update(Storage.BlobTargetOption.overrideUnlockedRetention(true));
storage.delete(tempBucket, retentionBlob);
storage.delete(tempBucket);
}
}
}