Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: external table definition parquet format options #2535

Merged
merged 6 commits into from Mar 7, 2023
Merged

fix: external table definition parquet format options #2535

merged 6 commits into from Mar 7, 2023

Conversation

jbeluch
Copy link
Contributor

@jbeluch jbeluch commented Feb 19, 2023

Fixes #2146

@jbeluch jbeluch requested review from a team and loferris February 19, 2023 01:08
@product-auto-label product-auto-label bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/java-bigquery API. labels Feb 19, 2023
@shollyman shollyman requested review from Neenu1995 and removed request for loferris February 22, 2023 22:18
@Neenu1995 Neenu1995 added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Feb 23, 2023
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Feb 23, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Feb 23, 2023
@gcf-owl-bot gcf-owl-bot bot requested a review from a team as a code owner February 23, 2023 20:53
@jbeluch
Copy link
Contributor Author

jbeluch commented Feb 27, 2023

Hi @Neenu1995 , is there an expected timeline for this type of change to be merged? Thanks.

@Neenu1995 Neenu1995 changed the title fix external table definition parquet format options fix: external table definition parquet format options Mar 2, 2023
@Neenu1995 Neenu1995 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Mar 2, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Mar 2, 2023
@Neenu1995
Copy link
Contributor

Tests fail:

java.lang.ClassCastException: com.google.cloud.bigquery.FormatOptions cannot be cast to com.google.cloud.bigquery.ParquetOptions
	at com.google.cloud.bigquery.ExternalTableDefinition.toExternalDataConfigurationPb(ExternalTableDefinition.java:315)
	at com.google.cloud.bigquery.ExternalTableDefinition.toPb(ExternalTableDefinition.java:283)
	at com.google.cloud.bigquery.TableInfo.toPb(TableInfo.java:511)
	at com.google.cloud.bigquery.BigQueryImpl.create(BigQueryImpl.java:290)
	at com.google.cloud.bigquery.it.ITBigQueryTest.testCreateExternalTableWithReferenceFileSchemaParquet(ITBigQueryTest.java:5175)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

@Neenu1995 Neenu1995 added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Mar 3, 2023
@jbeluch
Copy link
Contributor Author

jbeluch commented Mar 6, 2023

Tests fail:

java.lang.ClassCastException: com.google.cloud.bigquery.FormatOptions cannot be cast to com.google.cloud.bigquery.ParquetOptions
	at com.google.cloud.bigquery.ExternalTableDefinition.toExternalDataConfigurationPb(ExternalTableDefinition.java:315)
	at com.google.cloud.bigquery.ExternalTableDefinition.toPb(ExternalTableDefinition.java:283)
	at com.google.cloud.bigquery.TableInfo.toPb(TableInfo.java:511)
	at com.google.cloud.bigquery.BigQueryImpl.create(BigQueryImpl.java:290)
	at com.google.cloud.bigquery.it.ITBigQueryTest.testCreateExternalTableWithReferenceFileSchemaParquet(ITBigQueryTest.java:5175)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

thanks @Neenu1995, i missed running them the first time. ive made an additional fix so the IT test passes now. how should i handle squashing my commits? there was a bot commit on the branch so i didnt want to force push.

Neenu1995
Neenu1995 reviewed Mar 6, 2023
View reviewed changes
google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/FormatOptions.java
@@ -106,8 +106,8 @@ public static FormatOptions googleSheets() {
}

/** Default options for PARQUET format. */
public static FormatOptions parquet() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep the return type as is.
Changing it will break existing customers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed.

should i handle squashing commits myself or is it done at merge time?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jbeluch. The commits should be updated now.

@Neenu1995 Neenu1995 added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Mar 6, 2023
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 6, 2023
@yoshi-kokoro yoshi-kokoro removed kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Mar 6, 2023
@Neenu1995 Neenu1995 added kokoro:force-run Add this label to force Kokoro to re-run the tests. automerge Merge the pull request once unit tests and other checks pass. labels Mar 7, 2023
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Mar 7, 2023
@Neenu1995 Neenu1995 added the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 7, 2023
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Mar 7, 2023
@Neenu1995 Neenu1995 removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Mar 7, 2023
@Neenu1995 Neenu1995 merged commit eb45973 into googleapis:main Mar 7, 2023
17 checks passed
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Mar 7, 2023
@release-please release-please bot mentioned this pull request Mar 7, 2023
Merged
gcf-merge-on-green bot pushed a commit that referenced this pull request Mar 8, 2023
@release-please
chore(main): release 2.23.2 (#2556)
19385e6 
🤖 I have created a release *beep* *boop*
---


## [2.23.2](https://togithub.com/googleapis/java-bigquery/compare/v2.23.1...v2.23.2) (2023-03-07)


### Bug Fixes

* External table definition parquet format options ([#2535](https://togithub.com/googleapis/java-bigquery/issues/2535)) ([eb45973](https://togithub.com/googleapis/java-bigquery/commit/eb4597314fad72bbdb666a832f0f15f732f40817))


### Documentation

* Remove stale snippet comment ([#2555](https://togithub.com/googleapis/java-bigquery/issues/2555)) ([a71b1b2](https://togithub.com/googleapis/java-bigquery/commit/a71b1b2abb568bd1ed088f7bd4b77f93a68ec95e))

---
This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
This was referenced Mar 14, 2023
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Neenu1995 Neenu1995 Neenu1995 approved these changes

Assignees
No one assigned
Labels
api: bigquery Issues related to the googleapis/java-bigquery API. size: s Pull request size is small.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Parquet format options are ignored during BQ table creation
3 participants
@jbeluch @Neenu1995 @yoshi-kokoro