Skip to content

Commit

Permalink
feat: add schema aware stream writer (#2048)
Browse files Browse the repository at this point in the history
* feat: add schema aware stream writer

* [squash this commit] Fix clirr errors

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
  • Loading branch information
arturowczarek and gcf-owl-bot[bot] committed Apr 8, 2023
1 parent 8eda934 commit ad136b9
Show file tree
Hide file tree
Showing 10 changed files with 834 additions and 408 deletions.
2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -49,7 +49,7 @@ If you are using Maven without BOM, add this to your dependencies:
If you are using Gradle 5.x or later, add this to your dependencies:

```Groovy
implementation platform('com.google.cloud:libraries-bom:26.11.0')
implementation platform('com.google.cloud:libraries-bom:26.12.0')
implementation 'com.google.cloud:google-cloud-bigquerystorage'
```
Expand Down
Expand Up @@ -243,8 +243,8 @@ public String getStreamName() {
}

/**
* This exception is thrown from {@link JsonStreamWriter#append()} when the client side Json to
* Proto serializtion fails. It can also be thrown by the server in case rows contains invalid
* This exception is thrown from {@link SchemaAwareStreamWriter#append()} when the client side
* Proto serialization fails. It can also be thrown by the server in case rows contains invalid
* data. The exception contains a Map of indexes of faulty rows and the corresponding error
* message.
*/
Expand Down Expand Up @@ -362,16 +362,27 @@ protected InflightBytesLimitExceededException(String writerId, long currentLimit
currentLimit);
}
}

/**
* Input Json data has unknown field to the schema of the JsonStreamWriter. User can either turn
* on IgnoreUnknownFields option on the JsonStreamWriter, or if they don't want the error to be
* ignored, they should recreate the JsonStreamWriter with the updated table schema.
* This class is replaced by a generic one. It will be removed soon. Please use {@link
* DataHasUnknownFieldException}
*/
public static final class JsonDataHasUnknownFieldException extends IllegalArgumentException {
public static final class JsonDataHasUnknownFieldException extends DataHasUnknownFieldException {
protected JsonDataHasUnknownFieldException(String jsonFieldName) {
super(jsonFieldName);
}
}
/**
* Input data object has unknown field to the schema of the SchemaAwareStreamWriter. User can
* either turn on IgnoreUnknownFields option on the SchemaAwareStreamWriter, or if they don't want
* the error to be ignored, they should recreate the SchemaAwareStreamWriter with the updated
* table schema.
*/
public static class DataHasUnknownFieldException extends IllegalArgumentException {
private final String jsonFieldName;

protected JsonDataHasUnknownFieldException(String jsonFieldName) {
super(String.format("JSONObject has fields unknown to BigQuery: %s.", jsonFieldName));
public DataHasUnknownFieldException(String jsonFieldName) {
super(String.format("The source object has fields unknown to BigQuery: %s.", jsonFieldName));
this.jsonFieldName = jsonFieldName;
}

Expand Down

0 comments on commit ad136b9

Please sign in to comment.