Skip to content

Commit

Permalink
feat(dataflow): update the api
Browse files Browse the repository at this point in the history
#### dataflow:v1b3

The following keys were added:
- schemas.RuntimeUpdatableParams.properties.workerUtilizationHint (Total Keys: 2)
  • Loading branch information
yoshi-automation committed Dec 7, 2023
1 parent d9dcc8a commit ae34f86
Show file tree
Hide file tree
Showing 6 changed files with 119 additions and 96 deletions.
77 changes: 42 additions & 35 deletions docs/dyn/dataflow_v1b3.projects.jobs.html

Large diffs are not rendered by default.

11 changes: 6 additions & 5 deletions docs/dyn/dataflow_v1b3.projects.locations.flexTemplates.html
Expand Up @@ -229,7 +229,7 @@ <h3>Method Details</h3>
&quot;clientRequestId&quot;: &quot;A String&quot;, # The client&#x27;s unique identifier of the job, re-used across retried attempts. If this field is set, the service will ensure its uniqueness. The request to create a job will fail if the service has knowledge of a previously submitted job with the same client&#x27;s ID and job name. The caller may use this field to ensure idempotence of job creation across retried attempts to create a job. By default, the field is empty and, in that case, the service ignores it.
&quot;createTime&quot;: &quot;A String&quot;, # The timestamp when the job was initially created. Immutable and set by the Cloud Dataflow service.
&quot;createdFromSnapshotId&quot;: &quot;A String&quot;, # If this is specified, the job&#x27;s initial state is populated from the given snapshot.
&quot;currentState&quot;: &quot;A String&quot;, # The current state of the job. Jobs are created in the `JOB_STATE_STOPPED` state unless otherwise specified. A job in the `JOB_STATE_RUNNING` state may asynchronously enter a terminal state. After a job has reached a terminal state, no further state updates may be made. This field may be mutated by the Cloud Dataflow service; callers cannot mutate it.
&quot;currentState&quot;: &quot;A String&quot;, # The current state of the job. Jobs are created in the `JOB_STATE_STOPPED` state unless otherwise specified. A job in the `JOB_STATE_RUNNING` state may asynchronously enter a terminal state. After a job has reached a terminal state, no further state updates may be made. This field might be mutated by the Dataflow service; callers cannot mutate it.
&quot;currentStateTime&quot;: &quot;A String&quot;, # The timestamp associated with the current state.
&quot;environment&quot;: { # Describes the environment in which a Dataflow Job runs. # The environment for the job.
&quot;clusterManagerApiService&quot;: &quot;A String&quot;, # The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. &quot;compute.googleapis.com&quot;.
Expand Down Expand Up @@ -360,7 +360,7 @@ <h3>Method Details</h3>
},
},
},
&quot;id&quot;: &quot;A String&quot;, # The unique ID of this job. This field is set by the Cloud Dataflow service when the Job is created, and is immutable for the life of the job.
&quot;id&quot;: &quot;A String&quot;, # The unique ID of this job. This field is set by the Dataflow service when the job is created, and is immutable for the life of the job.
&quot;jobMetadata&quot;: { # Metadata available primarily for filtering jobs. Will be included in the ListJob response and Job SUMMARY view. # This field is populated by the Dataflow service to support filtering jobs by the metadata values provided here. Populated for ListJobs and all GetJob views SUMMARY and higher.
&quot;bigTableDetails&quot;: [ # Identification of a Cloud Bigtable source used in the Dataflow job.
{ # Metadata for a Cloud Bigtable connector used by the job.
Expand Down Expand Up @@ -421,7 +421,7 @@ <h3>Method Details</h3>
&quot;a_key&quot;: &quot;A String&quot;,
},
&quot;location&quot;: &quot;A String&quot;, # The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
&quot;name&quot;: &quot;A String&quot;, # The user-specified Cloud Dataflow job name. Only one Job with a given name can exist in a project within one region at any given time. Jobs in different regions can have the same name. If a caller attempts to create a Job with the same name as an already-existing Job, the attempt returns the existing Job. The name must match the regular expression `[a-z]([-a-z0-9]{0,1022}[a-z0-9])?`
&quot;name&quot;: &quot;A String&quot;, # The user-specified Dataflow job name. Only one active job with a given name can exist in a project within one region at any given time. Jobs in different regions can have the same name. If a caller attempts to create a job with the same name as an active job that already exists, the attempt returns the existing job. The name must match the regular expression `[a-z]([-a-z0-9]{0,1022}[a-z0-9])?`
&quot;pipelineDescription&quot;: { # A descriptive representation of submitted pipeline as well as the executed form. This data is provided by the Dataflow service for ease of visualizing the pipeline and interpreting Dataflow provided metrics. # Preliminary field: The format of this data may change at any time. A description of the user pipeline and stages through which it is executed. Created by Cloud Dataflow service. Only retrieved with JOB_VIEW_DESCRIPTION or JOB_VIEW_ALL.
&quot;displayData&quot;: [ # Pipeline level display data.
{ # Data provided with a pipeline or transform to provide descriptive info.
Expand Down Expand Up @@ -510,13 +510,14 @@ <h3>Method Details</h3>
],
&quot;stepNamesHash&quot;: &quot;A String&quot;, # A hash value of the submitted pipeline portable graph step names if exists.
},
&quot;projectId&quot;: &quot;A String&quot;, # The ID of the Cloud Platform project that the job belongs to.
&quot;projectId&quot;: &quot;A String&quot;, # The ID of the Google Cloud project that the job belongs to.
&quot;replaceJobId&quot;: &quot;A String&quot;, # If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job.
&quot;replacedByJobId&quot;: &quot;A String&quot;, # If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job.
&quot;requestedState&quot;: &quot;A String&quot;, # The job&#x27;s requested state. Applies to `UpdateJob` requests. Set `requested_state` with `UpdateJob` requests to switch between the states `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING`. You can also use `UpdateJob` requests to change a job&#x27;s state from `JOB_STATE_RUNNING` to `JOB_STATE_CANCELLED`, `JOB_STATE_DONE`, or `JOB_STATE_DRAINED`. These states irrevocably terminate the job if it hasn&#x27;t already reached a terminal state. This field has no effect on `CreateJob` requests.
&quot;runtimeUpdatableParams&quot;: { # Additional job parameters that can only be updated during runtime using the projects.jobs.update method. These fields have no effect when specified during job creation. # This field may ONLY be modified at runtime using the projects.jobs.update method to adjust job behavior. This field has no effect when specified at job creation.
&quot;maxNumWorkers&quot;: 42, # The maximum number of workers to cap autoscaling at. This field is currently only supported for Streaming Engine jobs.
&quot;minNumWorkers&quot;: 42, # The minimum number of workers to scale down to. This field is currently only supported for Streaming Engine jobs.
&quot;workerUtilizationHint&quot;: 3.14, # Target worker utilization, compared against the aggregate utilization of the worker pool by autoscaler, to determine upscaling and downscaling when absent other constraints such as backlog.
},
&quot;satisfiesPzi&quot;: True or False, # Output only. Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
&quot;satisfiesPzs&quot;: True or False, # Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests.
Expand Down Expand Up @@ -544,7 +545,7 @@ <h3>Method Details</h3>
&quot;transformNameMapping&quot;: { # The map of transform name prefixes of the job to be replaced to the corresponding name prefixes of the new job.
&quot;a_key&quot;: &quot;A String&quot;,
},
&quot;type&quot;: &quot;A String&quot;, # The type of Cloud Dataflow job.
&quot;type&quot;: &quot;A String&quot;, # The type of Dataflow job.
},
}</pre>
</div>
Expand Down

0 comments on commit ae34f86

Please sign in to comment.