Skip to content

Commit

Permalink
feat(dataflow): update the api
Browse files Browse the repository at this point in the history
#### dataflow:v1b3

The following keys were added:
- schemas.Environment.properties.streamingMode.type (Total Keys: 1)
  • Loading branch information
yoshi-automation committed Jan 4, 2024
1 parent 8d46d3c commit bac91ef
Show file tree
Hide file tree
Showing 6 changed files with 52 additions and 20 deletions.
21 changes: 14 additions & 7 deletions docs/dyn/dataflow_v1b3.projects.jobs.html
Original file line number Diff line number Diff line change
Expand Up @@ -190,8 +190,9 @@ <h3>Method Details</h3>
&quot;A String&quot;,
],
&quot;shuffleMode&quot;: &quot;A String&quot;, # Output only. The shuffle mode used for the job.
&quot;streamingMode&quot;: &quot;A String&quot;, # Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case.
&quot;tempStoragePrefix&quot;: &quot;A String&quot;, # The prefix of the resources the system should use for temporary storage. The system will append the suffix &quot;/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the new streaming engine billing model based on resource usage.
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the Streaming Engine resource-based billing model.
&quot;userAgent&quot;: { # A description of the process that generated the request.
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
},
Expand Down Expand Up @@ -545,8 +546,9 @@ <h3>Method Details</h3>
&quot;A String&quot;,
],
&quot;shuffleMode&quot;: &quot;A String&quot;, # Output only. The shuffle mode used for the job.
&quot;streamingMode&quot;: &quot;A String&quot;, # Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case.
&quot;tempStoragePrefix&quot;: &quot;A String&quot;, # The prefix of the resources the system should use for temporary storage. The system will append the suffix &quot;/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the new streaming engine billing model based on resource usage.
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the Streaming Engine resource-based billing model.
&quot;userAgent&quot;: { # A description of the process that generated the request.
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
},
Expand Down Expand Up @@ -884,8 +886,9 @@ <h3>Method Details</h3>
&quot;A String&quot;,
],
&quot;shuffleMode&quot;: &quot;A String&quot;, # Output only. The shuffle mode used for the job.
&quot;streamingMode&quot;: &quot;A String&quot;, # Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case.
&quot;tempStoragePrefix&quot;: &quot;A String&quot;, # The prefix of the resources the system should use for temporary storage. The system will append the suffix &quot;/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the new streaming engine billing model based on resource usage.
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the Streaming Engine resource-based billing model.
&quot;userAgent&quot;: { # A description of the process that generated the request.
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
},
Expand Down Expand Up @@ -1230,8 +1233,9 @@ <h3>Method Details</h3>
&quot;A String&quot;,
],
&quot;shuffleMode&quot;: &quot;A String&quot;, # Output only. The shuffle mode used for the job.
&quot;streamingMode&quot;: &quot;A String&quot;, # Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case.
&quot;tempStoragePrefix&quot;: &quot;A String&quot;, # The prefix of the resources the system should use for temporary storage. The system will append the suffix &quot;/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the new streaming engine billing model based on resource usage.
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the Streaming Engine resource-based billing model.
&quot;userAgent&quot;: { # A description of the process that generated the request.
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
},
Expand Down Expand Up @@ -1634,8 +1638,9 @@ <h3>Method Details</h3>
&quot;A String&quot;,
],
&quot;shuffleMode&quot;: &quot;A String&quot;, # Output only. The shuffle mode used for the job.
&quot;streamingMode&quot;: &quot;A String&quot;, # Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case.
&quot;tempStoragePrefix&quot;: &quot;A String&quot;, # The prefix of the resources the system should use for temporary storage. The system will append the suffix &quot;/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the new streaming engine billing model based on resource usage.
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the Streaming Engine resource-based billing model.
&quot;userAgent&quot;: { # A description of the process that generated the request.
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
},
Expand Down Expand Up @@ -2030,8 +2035,9 @@ <h3>Method Details</h3>
&quot;A String&quot;,
],
&quot;shuffleMode&quot;: &quot;A String&quot;, # Output only. The shuffle mode used for the job.
&quot;streamingMode&quot;: &quot;A String&quot;, # Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case.
&quot;tempStoragePrefix&quot;: &quot;A String&quot;, # The prefix of the resources the system should use for temporary storage. The system will append the suffix &quot;/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the new streaming engine billing model based on resource usage.
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the Streaming Engine resource-based billing model.
&quot;userAgent&quot;: { # A description of the process that generated the request.
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
},
Expand Down Expand Up @@ -2363,8 +2369,9 @@ <h3>Method Details</h3>
&quot;A String&quot;,
],
&quot;shuffleMode&quot;: &quot;A String&quot;, # Output only. The shuffle mode used for the job.
&quot;streamingMode&quot;: &quot;A String&quot;, # Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case.
&quot;tempStoragePrefix&quot;: &quot;A String&quot;, # The prefix of the resources the system should use for temporary storage. The system will append the suffix &quot;/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the new streaming engine billing model based on resource usage.
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the Streaming Engine resource-based billing model.
&quot;userAgent&quot;: { # A description of the process that generated the request.
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
},
Expand Down
3 changes: 2 additions & 1 deletion docs/dyn/dataflow_v1b3.projects.locations.flexTemplates.html
Original file line number Diff line number Diff line change
Expand Up @@ -258,8 +258,9 @@ <h3>Method Details</h3>
&quot;A String&quot;,
],
&quot;shuffleMode&quot;: &quot;A String&quot;, # Output only. The shuffle mode used for the job.
&quot;streamingMode&quot;: &quot;A String&quot;, # Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case.
&quot;tempStoragePrefix&quot;: &quot;A String&quot;, # The prefix of the resources the system should use for temporary storage. The system will append the suffix &quot;/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the new streaming engine billing model based on resource usage.
&quot;useStreamingEngineResourceBasedBilling&quot;: True or False, # Output only. Whether the job uses the Streaming Engine resource-based billing model.
&quot;userAgent&quot;: { # A description of the process that generated the request.
&quot;a_key&quot;: &quot;&quot;, # Properties of the object.
},
Expand Down

0 comments on commit bac91ef

Please sign in to comment.