Skip to content

Commit

Permalink
docs(bigquery): improve RowIterator docs and out of process paginatio…
Browse files Browse the repository at this point in the history
…n with Storage API (#8419)

Fixes #7285, fixes #8373, fixes #8173
  • Loading branch information
alvarowolfx committed Aug 15, 2023
1 parent 2bfdc95 commit 40f8f80
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 4 deletions.
2 changes: 2 additions & 0 deletions bigquery/bigquery.go
Expand Up @@ -101,6 +101,8 @@ func NewClient(ctx context.Context, projectID string, opts ...option.ClientOptio

// EnableStorageReadClient sets up Storage API connection to be used when fetching
// large datasets from tables, jobs or queries.
// Currently out of pagination methods like PageInfo().Token and RowIterator.StartIndex
// are not supported when the Storage API is enabled.
// Calling this method twice will return an error.
func (c *Client) EnableStorageReadClient(ctx context.Context, opts ...option.ClientOption) error {
if c.isStorageReadAvailable() {
Expand Down
4 changes: 3 additions & 1 deletion bigquery/doc.go
Expand Up @@ -104,7 +104,9 @@ To retrieve the job's results from the ID, first look up the Job:
}
Use the Job.Read method to obtain an iterator, and loop over the rows.
Query.Read is just a convenience method that combines Query.Run and Job.Read.
Calling Query.Read is preferred for queries with a relatively small result set,
as it will call BigQuery jobs.query API for a optimized query path. If the query
doesn't meet that criteria, the method will just combine Query.Run and Job.Read.
it, err = job.Read(ctx)
if err != nil {
Expand Down
16 changes: 13 additions & 3 deletions bigquery/iterator.go
Expand Up @@ -51,13 +51,21 @@ type RowIterator struct {
pf pageFetcher

// StartIndex can be set before the first call to Next. If PageInfo().Token
// is also set, StartIndex is ignored.
// is also set, StartIndex is ignored. If Storage API is enabled,
// StartIndex is also ignored because is not supported. IsAccelerated()
// method can be called to check if Storage API is enabled for the RowIterator.
StartIndex uint64

// The schema of the table. Available after the first call to Next.
// The schema of the table.
// In some scenarios it will only be available after the first
// call to Next(), like when a call to Query.Read uses
// the jobs.query API for an optimized query path.
Schema Schema

// The total number of rows in the result. Available after the first call to Next.
// The total number of rows in the result.
// In some scenarios it will only be available after the first
// call to Next(), like when a call to Query.Read uses
// the jobs.query API for an optimized query path.
// May be zero just after rows were inserted.
TotalRows uint64

Expand Down Expand Up @@ -169,6 +177,8 @@ func isStructPtr(x interface{}) bool {
}

// PageInfo supports pagination. See the google.golang.org/api/iterator package for details.
// Currently pagination is not supported when the Storage API is enabled. IsAccelerated()
// method can be called to check if Storage API is enabled for the RowIterator.
func (it *RowIterator) PageInfo() *iterator.PageInfo { return it.pageInfo }

func (it *RowIterator) fetch(pageSize int, pageToken string) (string, error) {
Expand Down

0 comments on commit 40f8f80

Please sign in to comment.