Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Copy all storage samples from java-docs-samples #1258

Merged
merged 79 commits into from
Mar 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
94947d9
Copy all storage samples from java-docs-samples
JesseLovelace Feb 16, 2022
144edbe
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 16, 2022
d0c8ddc
use gcloud-devel testing account
JesseLovelace Feb 16, 2022
cbddfac
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 16, 2022
08619fb
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 16, 2022
121249c
fix cleanup config
JesseLovelace Feb 16, 2022
9e565e1
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 16, 2022
05b8c05
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 16, 2022
750be2c
fix cfg
JesseLovelace Feb 16, 2022
4d7b06b
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 16, 2022
f11841f
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 16, 2022
0dcf62b
fix cfg and owlbot
JesseLovelace Feb 17, 2022
5c10df3
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 17, 2022
5f537ac
fix csek and request pays tests
JesseLovelace Feb 17, 2022
2648875
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 17, 2022
0ae066d
fix cfg and owlbot ignore
JesseLovelace Feb 17, 2022
dc6d5ff
Merge branch 'main' into movesamples
JesseLovelace Feb 17, 2022
8522d61
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 17, 2022
89bf36a
fix requester pays samples
JesseLovelace Feb 17, 2022
926791a
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 17, 2022
15a9735
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 17, 2022
b7481e3
fix download requester pays sample
JesseLovelace Feb 17, 2022
afd9a9e
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 17, 2022
95b8e37
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 17, 2022
2c32586
fix system out capture
JesseLovelace Feb 17, 2022
054e962
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 17, 2022
a65926a
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 17, 2022
9120eb4
fix disable requester pays sample
JesseLovelace Feb 17, 2022
2849557
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 17, 2022
7b38d19
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 17, 2022
6b94c36
fix label and kms tests
JesseLovelace Feb 17, 2022
b0fc440
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 17, 2022
08701db
add remaining samples and tests
JesseLovelace Feb 18, 2022
9d346e7
Merge branch 'main' into movesamples
JesseLovelace Feb 18, 2022
2e9cec7
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Feb 18, 2022
2287376
fix hmac cleanup
JesseLovelace Feb 18, 2022
2ef06db
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 18, 2022
0bad428
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Feb 18, 2022
dd10893
fix snippet output capture
JesseLovelace Feb 18, 2022
cc942e4
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 18, 2022
75fc930
fix standardOut reset
JesseLovelace Feb 18, 2022
e1c2e58
Fix standard out capture
JesseLovelace Feb 18, 2022
917ba15
increase retry backoff multiplier
JesseLovelace Feb 18, 2022
df7ab1c
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Feb 18, 2022
5eacbe9
fix UBLA sample and HMAC service account
JesseLovelace Feb 18, 2022
d044b70
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 18, 2022
09095cc
fix copyright header formatting
JesseLovelace Feb 18, 2022
bd6797e
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Feb 18, 2022
1c26ba9
fix checkstyle
JesseLovelace Feb 18, 2022
7728349
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Feb 18, 2022
ce8ba9f
add sleeps for hmac tests
JesseLovelace Feb 18, 2022
29bd318
use java docs samples testing account
JesseLovelace Feb 25, 2022
ce857ee
Merge branch 'main' into movesamples
JesseLovelace Feb 25, 2022
7ddb4c6
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Feb 25, 2022
a396dc8
Merge branch 'main' into movesamples
JesseLovelace Mar 2, 2022
d39aff9
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 2, 2022
4939c36
fix snapshot dependency
JesseLovelace Mar 2, 2022
f1d9a09
fix kms resource
JesseLovelace Mar 2, 2022
a3bfbb8
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 2, 2022
ee37dc5
fix hmac list test
JesseLovelace Mar 2, 2022
dd933da
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Mar 2, 2022
eb85d91
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 2, 2022
4f5df3d
add sleeps to fix 429 errors
JesseLovelace Mar 3, 2022
5699e6f
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Mar 3, 2022
4098e6d
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 3, 2022
8f194b4
fix hmac service account
JesseLovelace Mar 3, 2022
992e5ba
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Mar 3, 2022
4b8a352
Fix hmac service account
JesseLovelace Mar 3, 2022
983c953
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 3, 2022
793c562
fix standard out
JesseLovelace Mar 4, 2022
bf5fc12
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Mar 4, 2022
8810c2a
fix bucket retention test
JesseLovelace Mar 4, 2022
abe158b
fix checkstyle
JesseLovelace Mar 4, 2022
366fd81
Add comment about setout
JesseLovelace Mar 4, 2022
0431002
Merge branch 'main' into movesamples
JesseLovelace Mar 14, 2022
2e5730b
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 14, 2022
41ba1b9
fix acl sample comments
JesseLovelace Mar 15, 2022
132cd35
Merge branch 'movesamples' of github.com:googleapis/java-storage into…
JesseLovelace Mar 15, 2022
a202432
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .kokoro/nightly/samples.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,4 @@ env_vars: {
key: "IT_SERVICE_ACCOUNT_EMAIL"
value: "samples@java-docs-samples-testing.iam.gserviceaccount.com"
}

9 changes: 5 additions & 4 deletions .kokoro/presubmit/integration.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,12 @@ env_vars: {
}

env_vars: {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

owlbot did this, i didn't touch that file at all. it's in one of the "updates from owlbot" commits

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it's because of a mistake i made with the owlbot.py file at one point, i can fix it

key: "IT_SERVICE_ACCOUNT_EMAIL"
value: "it-service-account@gcloud-devel.iam.gserviceaccount.com"
key: "SECRET_MANAGER_KEYS"
value: "java-it-service-account"
}

env_vars: {
key: "SECRET_MANAGER_KEYS"
value: "java-it-service-account"
key: "IT_SERVICE_ACCOUNT_EMAIL"
value: "it-service-account@gcloud-devel.iam.gserviceaccount.com"
}

1 change: 1 addition & 0 deletions .kokoro/presubmit/samples.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,4 @@ env_vars: {
key: "IT_SERVICE_ACCOUNT_EMAIL"
value: "samples@java-docs-samples-testing.iam.gserviceaccount.com"
}

62 changes: 7 additions & 55 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,18 +48,17 @@ integration tests.

```bash
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account.json
export IT_SERVICE_ACCOUNT_EMAIL=my-service-account@my-project.iam.gserviceaccount.com
mvn -Penable-integration-tests clean verify
```

## Code Samples

Code Samples must be bundled in separate Maven modules, and guarded by a
Maven profile with the name `enable-samples`.
All code samples must be in compliance with the [java sample formatting guide][3].
Code Samples must be bundled in separate Maven modules.

The samples must be separate from the primary project for a few reasons:
1. Primary projects have a minimum Java version of Java 7 whereas samples have
a minimum Java version of Java 8. Due to this we need the ability to
1. Primary projects have a minimum Java version of Java 8 whereas samples can have
Java version of Java 11. Due to this we need the ability to
selectively exclude samples from a build run.
2. Many code samples depend on external GCP services and need
credentials to access the service.
Expand All @@ -69,39 +68,16 @@ The samples must be separate from the primary project for a few reasons:
### Building

```bash
mvn -Penable-samples clean verify
mvn clean verify
```

Some samples require access to GCP services and require a service account:

```bash
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account.json
mvn -Penable-samples clean verify
mvn clean verify
```

### Profile Config

1. To add samples in a profile to your Maven project, add the following to your
`pom.xml`

```xml
<project>
[...]
<profiles>
<profile>
<id>enable-samples</id>
<modules>
<module>sample</module>
</modules>
</profile>
</profiles>
[...]
</project>
```

2. [Activate](#profile-activation) the profile.
3. Define your samples in a normal Maven project in the `samples/` directory.

### Code Formatting

Code in this repo is formatted with
Expand All @@ -111,30 +87,6 @@ To run formatting on your project, you can run:
mvn com.coveo:fmt-maven-plugin:format
```

### Profile Activation

To include code samples when building and testing the project, enable the
`enable-samples` Maven profile.

#### Command line

To activate the Maven profile on the command line add `-Penable-samples` to your
Maven command.

#### Maven `settings.xml`

To activate the Maven profile in your `~/.m2/settings.xml` add an entry of
`enable-samples` following the instructions in [Active Profiles][2].

This method has the benefit of applying to all projects you build (and is
respected by IntelliJ IDEA) and is recommended if you are going to be
contributing samples to several projects.

#### IntelliJ IDEA

To activate the Maven Profile inside IntelliJ IDEA, follow the instructions in
[Activate Maven profiles][3] to activate `enable-samples`.

[1]: https://cloud.google.com/docs/authentication/getting-started#creating_a_service_account
[2]: https://maven.apache.org/settings.html#Active_Profiles
[3]: https://www.jetbrains.com/help/idea/work-with-maven-profiles.html#activate_maven_profiles
[3]: https://github.com/GoogleCloudPlatform/java-docs-samples/blob/main/SAMPLE_FORMAT.md
87 changes: 86 additions & 1 deletion README.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -3017,7 +3017,7 @@ private void retentionPolicyLockRequesterPays(boolean requesterPays)
storage.update(bucketInfo, Storage.BucketTargetOption.userProject(projectId));
assertFalse(updateBucket.requesterPays());
}
RemoteStorageHelper.forceDelete(storage, bucketName, 5, TimeUnit.SECONDS);
RemoteStorageHelper.forceDelete(storage, bucketName, 5, TimeUnit.SECONDS, projectId);
}
}

Expand Down
12 changes: 12 additions & 0 deletions samples/install-without-bom/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,18 @@
<version>1.1.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.13</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>4.5.13</version>
<scope>test</scope>
</dependency>
</dependencies>

<!-- compile and run all snippet tests -->
Expand Down
8 changes: 7 additions & 1 deletion samples/snapshot/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-storage</artifactId>
<version>2.4.5</version>
<version>2.4.6-SNAPSHOT</version>
</dependency>

<dependency>
Expand All @@ -43,6 +43,12 @@
<version>1.1.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>4.5.13</version>
<scope>test</scope>
</dependency>
</dependencies>
<!-- {x-version-update-end} -->

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
/*
* Copyright 2020 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.storage;

// [START storage_generate_signed_post_policy_v4]

import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.PostPolicyV4;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
import java.util.Map;
import java.util.concurrent.TimeUnit;

public class GenerateSignedPostPolicyV4 {
/**
* Generating a signed POST policy requires Credentials which implement ServiceAccountSigner.
* These can be set explicitly using the Storage.PostPolicyV4Option.signWith(ServiceAccountSigner)
* option. If you don't, you could also pass a service account signer to StorageOptions, i.e.
* StorageOptions().newBuilder().setCredentials(ServiceAccountSignerCredentials). In this example,
* neither of these options are used, which means the following code only works when the
* credentials are defined via the environment variable GOOGLE_APPLICATION_CREDENTIALS, and those
* credentials are authorized to sign a policy. See the documentation for
* Storage.generateSignedPostPolicyV4 for more details.
*/
public static void generateSignedPostPolicyV4(
String projectId, String bucketName, String blobName) {
// The ID of your GCP project
// String projectId = "your-project-id";

// The ID of the GCS bucket to upload to
// String bucketName = "your-bucket-name"

// The name to give the object uploaded to GCS
// String blobName = "your-object-name"

Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();

PostPolicyV4.PostFieldsV4 fields =
PostPolicyV4.PostFieldsV4.newBuilder().setCustomMetadataField("test", "data").build();

PostPolicyV4 policy =
storage.generateSignedPostPolicyV4(
BlobInfo.newBuilder(bucketName, blobName).build(), 10, TimeUnit.MINUTES, fields);

StringBuilder htmlForm =
new StringBuilder(
"<form action='"
+ policy.getUrl()
+ "' method='POST' enctype='multipart/form-data'>\n");
for (Map.Entry<String, String> entry : policy.getFields().entrySet()) {
htmlForm.append(
" <input name='"
+ entry.getKey()
+ "' value='"
+ entry.getValue()
+ "' type='hidden' />\n");
}
htmlForm.append(" <input type='file' name='file'/><br />\n");
htmlForm.append(" <input type='submit' value='Upload File'/><br />\n");
htmlForm.append("</form>\n");

System.out.println(
"You can use the following HTML form to upload an object to bucket "
+ bucketName
+ " for the next ten minutes:");
System.out.println(htmlForm.toString());
}
}
// [END storage_generate_signed_post_policy_v4]
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*
* Copyright 2020 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.storage;

// [START storage_get_service_account]
import com.google.cloud.storage.ServiceAccount;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

public class GetServiceAccount {
public static void getServiceAccount(String projectId) {
// The ID of your GCP project
// String projectId = "your-project-id";

Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();
ServiceAccount serviceAccount = storage.getServiceAccount(projectId);
System.out.println(
"The GCS service account for project " + projectId + " is: " + serviceAccount.getEmail());
}
}
// [END storage_get_service_account]
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ public class AddBucketDefaultOwner {

public static void addBucketDefaultOwner(String bucketName, String userEmail) {

// The ID to give your GCS bucket
// The ID of your GCS bucket
// String bucketName = "your-unique-bucket-name";

// Email of the user you wish to add as a default owner
// The email of the user you wish to add as a default owner
// String userEmail = "someuser@domain.com"

Storage storage = StorageOptions.newBuilder().build().getService();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
/*
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.storage.bucket;

// [START storage_add_bucket_conditional_iam_binding]

import com.google.cloud.Binding;
import com.google.cloud.Condition;
import com.google.cloud.Policy;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

public class AddBucketIamConditionalBinding {
/** Example of adding a conditional binding to the Bucket-level IAM */
public static void addBucketIamConditionalBinding(String projectId, String bucketName) {
// The ID of your GCP project
// String projectId = "your-project-id";

// The ID of your GCS bucket
// String bucketName = "your-unique-bucket-name";

// For more information please read:
// https://cloud.google.com/storage/docs/access-control/iam
Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();

Policy originalPolicy =
storage.getIamPolicy(bucketName, Storage.BucketSourceOption.requestedPolicyVersion(3));

String role = "roles/storage.objectViewer";
String member = "group:example@google.com";

// Create a condition
String conditionTitle = "Title";
String conditionDescription = "Description";
String conditionExpression =
"resource.name.startsWith(\"projects/_/buckets/bucket-name/objects/prefix-a-\")";
Condition.Builder conditionBuilder = Condition.newBuilder();
conditionBuilder.setTitle(conditionTitle);
conditionBuilder.setDescription(conditionDescription);
conditionBuilder.setExpression(conditionExpression);

// getBindingsList() returns an ImmutableList, we copy over to an ArrayList so it's mutable
List<Binding> bindings = new ArrayList(originalPolicy.getBindingsList());

// Add condition to a binding
Binding.Builder newBindingBuilder =
Binding.newBuilder()
.setRole(role)
.setMembers(Arrays.asList(member))
.setCondition(conditionBuilder.build());
bindings.add(newBindingBuilder.build());

// Update policy with new conditional binding
Policy.Builder updatedPolicyBuilder = originalPolicy.toBuilder();
updatedPolicyBuilder.setBindings(bindings).setVersion(3);

storage.setIamPolicy(bucketName, updatedPolicyBuilder.build());

System.out.printf(
"Added %s with role %s to %s with condition %s %s %s\n",
member, role, bucketName, conditionTitle, conditionDescription, conditionExpression);
}
}
// [END storage_add_bucket_conditional_iam_binding]