Create a method to generate OAuth tokens #644

hectorcast-db · 2024-05-16T08:59:02Z

Changes

Add method to get OAuth tokens

Tests

make test run locally
make fmt applied
relevant integration tests applied
Manual test (cannot be run as integration tests due to limitations in the current infrastructure setup)

def test():
    w = WorkspaceClient(profile='DEFAULT')
    auth_details = f'"type":"workspace_permission","object_type":"serving-endpoints","object_path":"/serving-endpoints/REDACTED","actions":["query_inference_endpoint"]'
    auth_details = "[{" + auth_details + "}]"
    t = w.api_client.get_oauth_token(auth_details)
    print(t)

Result:

Token(access_token='REDACTED', token_type='Bearer', refresh_token=None, expiry=datetime.datetime(2024, 5, 16, 11, 9, 8, 221008))

github-actions · 2024-05-16T08:59:53Z

This PR breaks backwards compatibility for databrickslabs/blueprint downstream. See build logs for more details.

_{Running from downstreams #131}

mgyucht

Some suggestions, but coming along nicely

mgyucht · 2024-05-16T13:12:34Z

databricks/sdk/core.py

@@ -108,6 +109,21 @@ def flatten_dict(d: Dict[str, Any]) -> Dict[str, Any]:
        flattened = dict(flatten_dict(with_fixed_bools))
        return flattened

+    def get_oauth_token(self, auth_details: str) -> Token:


Does it make sense for some of this code to live in oauth.py?

I did a few changes to reuse some function from oauth.py (pending to push). However, this method still makes sense for 2 reasons:

Config is not exposed int he cli and we need it

Even after exposing the config, we can't pass it to oauth.py or we create circular dependencies.
So having this here is helpful because it simplifies things. I could move the creation of params inside oauth.py.

databricks/sdk/core.py

mgyucht · 2024-05-16T13:14:10Z

databricks/sdk/credentials_provider.py

+    def oauth_token(self) -> Token:
+        ...
+
+
 class CredentialsProvider(abc.ABC):


Shall we rename CredentialsProvider to CredentialsStrategy here as well?

databricks/sdk/credentials_provider.py

github-actions · 2024-05-17T12:09:10Z

This PR breaks backwards compatibility for databrickslabs/ucx downstream. See build logs for more details.

_{Running from downstreams #131}

codecov-commenter · 2024-05-17T12:12:19Z

Codecov Report

Attention: Patch coverage is 77.87611% with 25 lines in your changes are missing coverage. Please review.

Project coverage is 57.67%. Comparing base (b13042b) to head (f7d920e).

Files	Patch %	Lines
databricks/sdk/credentials_provider.py	79.26%	17 Missing ⚠️
databricks/sdk/core.py	50.00%	6 Missing ⚠️
databricks/sdk/config.py	83.33%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #644      +/-   ##
==========================================
+ Coverage   57.66%   57.67%   +0.01%     
==========================================
  Files          48       48              
  Lines       32680    32742      +62     
==========================================
+ Hits        18844    18885      +41     
- Misses      13836    13857      +21

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mgyucht

Two small remarks, otherwise this looks good! Feel free to click the merge button when ready.

mgyucht · 2024-05-17T13:32:22Z

.codegen/__init__.py.tmpl

@@ -46,11 +46,11 @@ class WorkspaceClient:
                 debug_headers: bool = None,
                 product="unknown",
                 product_version="0.0.0",
-                 credentials_provider: CredentialsProvider = None,


Let's respect credentials_provider here as well, like what you do in Config

mgyucht · 2024-05-17T13:33:26Z

databricks/sdk/config.py

                 product="unknown",
                 product_version="0.0.0",
                 clock: Clock = None,
                 **kwargs):
        self._header_factory = None
        self._inner = {}
        self._user_agent_other_info = []
-        self._credentials_provider = credentials_provider if credentials_provider else DefaultCredentials()
+        if credentials_strategy and credentials_provider:
+            raise ValueError("When providing `credentials_strategy` field, `credential_provider` cannot be specified.")


Suggestion: let's log a warning here if credentials_provider attribute is set.

### Backward incompatible changes * `CredentialsProvider` class renamed to `CredentialsStrategy` and `HeaderFactory` class renamed to `CredentialsProvider` ### Improvements and new features * Better error message when private link enabled workspaces reject requests ([#647](#647)). * Create a method to generate OAuth tokens ([#644](#644)). API Changes: * Changed `list()` method for [w.connections](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/connections.html) workspace-level service to require request of `databricks.sdk.service.catalog.ListConnectionsRequest` dataclass. * Removed [w.lakehouse_monitors](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/lakehouse_monitors.html) workspace-level service. * Added [w.quality_monitors](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/quality_monitors.html) workspace-level service. * Renamed `databricks.sdk.service.catalog.DeleteLakehouseMonitorRequest` dataclass to `databricks.sdk.service.catalog.DeleteQualityMonitorRequest`. * Changed `schema_name` field for `databricks.sdk.service.catalog.DisableRequest` to `str` dataclass. * Removed `databricks.sdk.service.catalog.DisableSchemaName` dataclass. * Changed `schema_name` field for `databricks.sdk.service.catalog.EnableRequest` to `str` dataclass. * Removed `databricks.sdk.service.catalog.EnableSchemaName` dataclass. * Renamed `databricks.sdk.service.catalog.GetLakehouseMonitorRequest` dataclass to `databricks.sdk.service.catalog.GetQualityMonitorRequest`. * Added `next_page_token` field for `databricks.sdk.service.catalog.ListConnectionsResponse`. * Added `dashboard_id` field for `databricks.sdk.service.catalog.UpdateMonitor`. * Added `databricks.sdk.service.catalog.ListConnectionsRequest` dataclass. * Added `databricks.sdk.service.catalog.MonitorRefreshListResponse` dataclass. * Changed `cluster_status()` method for [w.libraries](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/libraries.html) workspace-level service to return `databricks.sdk.service.compute.ClusterLibraryStatuses` dataclass. * Removed `cluster_source` field for `databricks.sdk.service.compute.ClusterAttributes`. * Changed `spec` and `cluster_source` fields for `databricks.sdk.service.compute.ClusterDetails` to `databricks.sdk.service.compute.ClusterSpec` dataclass. * Removed `cluster_source` field for `databricks.sdk.service.compute.ClusterSpec`. * Removed `databricks.sdk.service.compute.ClusterStatusResponse` dataclass. * Removed `cluster_source` field for `databricks.sdk.service.compute.CreateCluster`. * Removed `clone_from` and `cluster_source` fields for `databricks.sdk.service.compute.EditCluster`. * Removed `sort_by_spec` field for `databricks.sdk.service.marketplace.ListListingsRequest`. * Added `is_ascending` and `sort_by` fields for `databricks.sdk.service.marketplace.ListListingsRequest`. * Added `is_ascending` field for `databricks.sdk.service.marketplace.SearchListingsRequest`. * Removed `databricks.sdk.service.marketplace.SortBySpec` dataclass. * Removed `databricks.sdk.service.marketplace.SortOrder` dataclass. * Added `gateway_definition` field for `databricks.sdk.service.pipelines.CreatePipeline`. * Added `gateway_definition` field for `databricks.sdk.service.pipelines.EditPipeline`. * Added `table_configuration` field for `databricks.sdk.service.pipelines.ManagedIngestionPipelineDefinition`. * Added `gateway_definition` field for `databricks.sdk.service.pipelines.PipelineSpec`. * Added `table_configuration` field for `databricks.sdk.service.pipelines.SchemaSpec`. * Added `table_configuration` field for `databricks.sdk.service.pipelines.TableSpec`. * Added `databricks.sdk.service.pipelines.IngestionGatewayPipelineDefinition` dataclass. * Added `databricks.sdk.service.pipelines.TableSpecificConfig` dataclass. * Added `databricks.sdk.service.pipelines.TableSpecificConfigScdType` dataclass. * Added `deployment_artifacts` field for `databricks.sdk.service.serving.AppDeployment`. * Added `route_optimized` field for `databricks.sdk.service.serving.CreateServingEndpoint`. * Added `contents` field for `databricks.sdk.service.serving.ExportMetricsResponse`. * Changed `openai_api_key` field for `databricks.sdk.service.serving.OpenAiConfig` to no longer be required. * Added `microsoft_entra_client_id`, `microsoft_entra_client_secret` and `microsoft_entra_tenant_id` fields for `databricks.sdk.service.serving.OpenAiConfig`. * Added `endpoint_url` and `route_optimized` fields for `databricks.sdk.service.serving.ServingEndpointDetailed`. * Added `databricks.sdk.service.serving.AppDeploymentArtifacts` dataclass. * Added `storage_root` field for `databricks.sdk.service.sharing.CreateShare`. * Added `storage_location` and `storage_root` fields for `databricks.sdk.service.sharing.ShareInfo`. * Added `storage_root` field for `databricks.sdk.service.sharing.UpdateShare`. * Added `scan_index()` method for [w.vector_search_indexes](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/vector_search_indexes.html) workspace-level service. * Added `embedding_writeback_table` field for `databricks.sdk.service.vectorsearch.DeltaSyncVectorIndexSpecRequest`. * Added `embedding_writeback_table` field for `databricks.sdk.service.vectorsearch.DeltaSyncVectorIndexSpecResponse`. * Added `databricks.sdk.service.vectorsearch.ListValue` dataclass. * Added `databricks.sdk.service.vectorsearch.MapStringValueEntry` dataclass. * Added `databricks.sdk.service.vectorsearch.ScanVectorIndexRequest` dataclass. * Added `databricks.sdk.service.vectorsearch.ScanVectorIndexResponse` dataclass. * Added `databricks.sdk.service.vectorsearch.Struct` dataclass. * Added `databricks.sdk.service.vectorsearch.Value` dataclass. OpenAPI SHA: 7eb5ad9a2ed3e3f1055968a2d1014ac92c06fe92, Date: 2024-05-21

This reverts commit db1f4ae.

## Changes This reverts commit db1f4ae. Azure CredentialsProvider is impacted by the change: https://github.com/databricks-eng/eng-dev-ecosystem/actions/runs/9187156296/job/25264245409 Reverting to unblock the release ## Tests - [X] `make test` run locally - [X] `make fmt` applied - [X] relevant integration tests applied

This reverts commit 9c5fae7.

…" (#655) This reverts commit 9c5fae7. ## Changes Introduce again the method to generate OAuth tokens, including fixes for impacted auth types ## Tests - [x] `make test` run locally - [x] `make fmt` applied - [x] relevant integration tests applied

hectorcast-db added 6 commits May 15, 2024 15:18

WIP

5eb344f

Fixes

2166d73

fixes

78643cc

fixes

8ab2c68

Rename

7a0144f

Comment

0db2121

hectorcast-db requested a review from mgyucht May 16, 2024 08:59

hectorcast-db changed the title ~~Add method to get Oauth tokens~~ Create a method to generate OAuth tokens May 16, 2024

mgyucht reviewed May 16, 2024

View reviewed changes

hectorcast-db added 2 commits May 16, 2024 16:39

Comments

3b527eb

Rename

b982266

Comment

613c310

Simplify

0f4469a

mgyucht approved these changes May 17, 2024

View reviewed changes

hectorcast-db added 3 commits May 17, 2024 16:29

Comments

c867145

Merge branch 'main' into generate-oauth-token

604f5c2

fmt

f7d920e

hectorcast-db added this pull request to the merge queue May 21, 2024

Merged via the queue into main with commit db1f4ae May 21, 2024
7 of 9 checks passed

hectorcast-db deleted the generate-oauth-token branch May 21, 2024 07:30

hectorcast-db mentioned this pull request May 22, 2024

Release v0.28.0 #652

Merged

hectorcast-db added a commit that referenced this pull request May 22, 2024

Revert "Create a method to generate OAuth tokens (#644)"

4f2cd55

This reverts commit db1f4ae.

hectorcast-db added a commit that referenced this pull request May 24, 2024

Revert "Revert "Create a method to generate OAuth tokens (#644)" (#653)"

34ac529

This reverts commit 9c5fae7.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a method to generate OAuth tokens #644

Create a method to generate OAuth tokens #644

hectorcast-db commented May 16, 2024 •

edited

github-actions bot commented May 16, 2024 •

edited

mgyucht left a comment

mgyucht May 16, 2024

hectorcast-db May 17, 2024 •

edited

mgyucht May 16, 2024

github-actions bot commented May 17, 2024 •

edited

codecov-commenter commented May 17, 2024 •

edited

mgyucht left a comment

mgyucht May 17, 2024

mgyucht May 17, 2024

Create a method to generate OAuth tokens #644

Create a method to generate OAuth tokens #644

Conversation

hectorcast-db commented May 16, 2024 • edited

Changes

Tests

github-actions bot commented May 16, 2024 • edited

mgyucht left a comment

Choose a reason for hiding this comment

mgyucht May 16, 2024

Choose a reason for hiding this comment

hectorcast-db May 17, 2024 • edited

Choose a reason for hiding this comment

mgyucht May 16, 2024

Choose a reason for hiding this comment

github-actions bot commented May 17, 2024 • edited

codecov-commenter commented May 17, 2024 • edited

Codecov Report

mgyucht left a comment

Choose a reason for hiding this comment

mgyucht May 17, 2024

Choose a reason for hiding this comment

mgyucht May 17, 2024

Choose a reason for hiding this comment

hectorcast-db commented May 16, 2024 •

edited

github-actions bot commented May 16, 2024 •

edited

hectorcast-db May 17, 2024 •

edited

github-actions bot commented May 17, 2024 •

edited

codecov-commenter commented May 17, 2024 •

edited