Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add telemetry support for mlflow models #4674

Merged
merged 4 commits into from May 13, 2024
Merged

Add telemetry support for mlflow models #4674

merged 4 commits into from May 13, 2024

Conversation

jiapinw
Copy link
Collaborator

@jiapinw jiapinw commented May 10, 2024 •
edited

Issue #, if available:

Description of changes:

  • Add telemetry support for mlflow models
  • Fix typo and regex patterns

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

  • I have read the CONTRIBUTING doc
  • I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
  • I used the commit message format described in CONTRIBUTING
  • I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
  • I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

  • I have added tests that prove my fix is effective or that my feature works (if appropriate)
  • I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
  • I have checked that my tests are not configured for a specific region or account (if appropriate)
  • I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jiapinw jiapinw marked this pull request as ready for review May 13, 2024 16:31
@jiapinw jiapinw requested a review from a team as a code owner May 13, 2024 16:31
@jiapinw jiapinw requested review from pintaoz-aws and removed request for a team May 13, 2024 16:31
@mufaddal-rohawala mufaddal-rohawala merged commit 2f2be05 into aws:master May 13, 2024
10 of 11 checks passed
benieric added a commit that referenced this pull request May 16, 2024
@benieric @Captainia
@nskool @jessicazhu3 @jmahlik @akrishna1995 @haixiw @zhaoqizqwang @knikure @keerthanvasist @SuhitK @dependabot @sagemaker-bot @jiapinw @makungaj1 @prateekdesai04 @mufaddal-rohawala @samruds @evakravi
Master benchmark feature (#4672)
2b71ab4 
* fix: mainline alt config parsing (#4602)

* fix: parsing

* fix: commit tests

* fix: types

* updated

* fix

* Add Triton v24.03 URI (#4605)

Co-authored-by: Nikhil Kulkarni <nikhilsk@amazon.com>

* feature: support session tag chaining for training job (#4596)

* feature: support session tag chaining for training job

* fix: resolve typo

* fix: resolve typo and build failure

* fix: resolve typo and unit test failure

---------

Co-authored-by: Jessica Zhu <jessicazhu3@106775307+jessicazhu3@users.noreply.github.com>

* prepare release v2.217.0

* update development version to v2.217.1.dev0

* fix: properly close files in lineage queries and tests (#4587)

Closes #4458

* feature: set default allow_pickle param to False (#4557)

* breaking: set default allow_pickle param to False

* breaking: fix unit tests and linting

NumpyDeserializer will not allow deserialization
unless allow_pickle flag is set to True explicitly

* fix: black-check

---------

Co-authored-by: Ashwin Krishna <ashwikri@amazon.com>

* Fix:invalid component error with new metadata (#4634)

* fix: invalid component name

* tests

* format

* fix vulnerable model integ tests llama 2

* updated

* fix: training dataset location

* prepare release v2.218.0

* update development version to v2.218.1.dev0

* chore: update skipped flaky tests (#4644)

* Update skipped flaky tests

* flake8

* format

* format

* chore: release tgi 2.0.1 (#4642)

* chore: release tgi 2.0.1

* minor fix

---------

Co-authored-by: Zhaoqi <52220743+zhaoqizqwang@users.noreply.github.com>

* fix: Fix UserAgent logging in Python SDK (#4647)

* prepare release v2.218.1

* update development version to v2.218.2.dev0

* feature: allow choosing js payload by alias in private method

* Updates for SMP v2.3.1 (#4660)

Co-authored-by: Suhit Kodgule <skodgule@amazon.com>

* chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /doc (#4655)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.3...3.1.4)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump tqdm from 4.66.2 to 4.66.3 in /tests/data/serve_resources/mlflow/pytorch (#4650)

Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.2 to 4.66.3.
- [Release notes](https://github.com/tqdm/tqdm/releases)
- [Commits](tqdm/tqdm@v4.66.2...v4.66.3)

---
updated-dependencies:
- dependency-name: tqdm
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /requirements/extras (#4654)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.3...3.1.4)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* prepare release v2.219.0

* update development version to v2.219.1.dev0

* fix: skip flakey tests pending investigation (#4667)

* change: update image_uri_configs  05-09-2024 07:17:41 PST

* Add tensorflow_serving support for mlflow models and enable lineage tracking for mlflow models (#4662)

* Initial commit for tensorflow_serving support of MLflow

* Add integ tests for mlflow tf_serving

* fix style issues

* remove unused attributes from tf builder

* Add deep ping for tf_serving local mode

* Initial commit for lineage impl

* Initial commit for tensorflow_serving support of MLflow

* Add integ tests for mlflow tf_serving

* fix style issues

* remove unused attributes from tf builder

* Add deep ping for tf_serving local mode

* Add integ tests and uts

* fix local mode for tf_serving

* Allow lineage tracking only in sagemaker endpoint mode

* fix regex pattern

* fix style issues

* fix regex pattern and hard coded py version in ut

* fix missing session

* Resolve pr comments and fix regex for mlflow registry and ids

* fix: model builder race condition on sagemaker session (#4673)

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* feat: Add telemetry support for mlflow models (#4674)

* Initial commit for telemetry support

* Fix style issues and add more logger messages

* fix value error messages in ut

* feat: add new images for HF TGI release (#4677)

* chore: add new images for HF TGI release

* test

* feature: AutoGluon 1.1.0 image_uris update (#4679)

Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-154.us-west-2.compute.internal>

* change: add debug logs to workflow container dist creation (#4682)

* prepare release v2.220.0

* update development version to v2.220.1.dev0

* fix: Image URI should take precedence for HF models (#4684)

* Fix: Image URI should take precedence for HF models

* Fix formatting

* Fix formatting

* Fix formatting

* Increase coverage -  UT pass

* feat: support config_name in all JumpStart interfaces (#4583) (#4607)

* add-config-name

* address comments

* updates for set config

* docstyle

* updates

* fix

* format

* format

* remove tests

* Add ReadOnly APIs (#4606)

* Add ReadOnly APIs

* Resolving PR review comments

* Resolve PR review comments

* Refactoring

* Refactoring

* Add Caching

* Refactore

* Resolving conflicts

* Add Unit Tests

* Fix Unit Tests

* Fix unit tests

* Fix UT

* Refactoring

* Fix Integ tests

* refactoring after Notebook testing

* Fix code styles

---------

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* feat: tag JumpStart resource with config names (#4608)

* tag config name

* format

* resolving comments

* format

* format

* update

* fix

* format

* updates inference component config name

* fix: tests

* ModelBuilder: Add functionalities to get and set deployment config. (#4614)

* Add funtionalities to get and set deployment config

* Resolve PR comments

* ModelBuilder-JS

* Add Unit tests

* Refactoring

* Testing with Notebook

* Test backward compatibility

* Remove Accelerated column if all not enabled

* Fix docstring

* Resolved PR Review comments

* Docstring

* increase code coverage

---------

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* Benchmark feature v2 (#4618)

* Add funtionalities to get and set deployment config

* Resolve PR comments

* ModelBuilder-JS

* Add Unit tests

* Refactoring

* Testing with Notebook

* Test backward compatibility

* Remove Accelerated column if all not enabled

* Fix docstring

* Resolved PR Review comments

* Docstring

* increase code coverage

* Testing fix with Notebook

* Only fetch instance rate metrics if not present

* Increase code coverage

---------

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* fix: populate default config name to model (#4617)

* fix: populate default config name to model

* update condition

* fix

* format

* flake8

* fix tests

* fix coverage

* temporarily skip integ test vulnerbility

* fix tolerate attach method

* format

* fix predictor

* format

* Fix fetch instance rate bug (#4624)

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* chore: require config name and instance type in set_deployment_config (#4625)

* require config_name and instance_type in set config

* docstring

* add supported instance types check

* add more tests

* format

* fix tests

* Deployment Configs - Follow-ups (#4626)

* Init Deployment configs outside Model init.

* Testing with NB

* Testing with NB-V2

* Refactoring, NB testing

* NB Testing and Refactoring

* Testing

* Refactoring

* Testing with NB

* Debug

* Debug display API

* Debug with NB

* Testing with NB

* Refactoring

* Refactoring

* Refactoring and NB testing

* Testing with NB

* Refactoring

* Prefix instance type with ml

* Fix unit tests

---------

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* fix: use different separator to flatten dict (#4629)

* Use separate tags for inference and training configs (#4635)

* Use separate tags for inference and training

* format

* format

* format

* format

* Add supported inference and incremental training configs (#4637)

* supported inference configs

* add tests

* format

* tests

* tests

* address comments

* format and address comments

* updates

* formt

* format

* Benchmark feature fixes (#4632)

* Filter down Benchmark Metrics

* Filter down Benchmark Metrics

* Testing NB

* Testing MB

* Testing

* Refactoring

* Unit tests

* Display instance type first, and instance rate last

* Display unbalanced metrics

* Testing with NB

* Testing with NB

* Debug

* Debug

* Testing with NB

* Testing with NB

* Testing with NB

* Refactoring

* Refactoring

* Refactoring

* Unit tests

* Custom lru

* Custom lru

* Custom lru

* Custom lru

* Custom lru

* Custom lru

* Custom lru

* Custom lru

* Custom lru

* Custom lru

* Refactoring

* Debug

* Config ranking

* Debug

* Debug

* Debug

* Debug

* Debug

* Ranking

* Ranking-Debug

* Ranking-Debug

* Ranking-Debug

* Ranking-Debug

* Ranking-Debug

* Ranking-Debug

* Debug

* Debug

* Debug

* Debug

* Refactoring

* Contact JumpStart team to fix flaky test. test_list_jumpstart_models_script_filter

---------

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* fix: typo and merge with master branch (#4649)

* Merge master into benchmark feature (#4652)

* Merge master into master-benchmark-feature (#4656)

* Master benchmark feature (#4658)

* Remove duplicate line in types.py

* Remove duplicate lines

* Remove duplicate lines

* Remove duplicate lines

* Remove duplicate lines

* fix unit test

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Haotian An <33510317+Captainia@users.noreply.github.com>
Co-authored-by: Nikhil Kulkarni <knikhil29@gmail.com>
Co-authored-by: Nikhil Kulkarni <nikhilsk@amazon.com>
Co-authored-by: jessicazhu3 <106775307+jessicazhu3@users.noreply.github.com>
Co-authored-by: Jessica Zhu <jessicazhu3@106775307+jessicazhu3@users.noreply.github.com>
Co-authored-by: ci <ci>
Co-authored-by: Justin <justinm088@hotmail.com>
Co-authored-by: ASHWIN KRISHNA <38850354+akrishna1995@users.noreply.github.com>
Co-authored-by: Ashwin Krishna <ashwikri@amazon.com>
Co-authored-by: Haixin Wang <98612668+haixiw@users.noreply.github.com>
Co-authored-by: Zhaoqi <52220743+zhaoqizqwang@users.noreply.github.com>
Co-authored-by: Kalyani Nikure <110067132+knikure@users.noreply.github.com>
Co-authored-by: Keerthan Vasist <kvasist@amazon.com>
Co-authored-by: SuhitK <kodgule.suhit@gmail.com>
Co-authored-by: Suhit Kodgule <skodgule@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sagemaker-bot <sagemaker-bot@amazon.com>
Co-authored-by: jiapinw <95885824+jiapinw@users.noreply.github.com>
Co-authored-by: Jonathan Makunga <54963715+makungaj1@users.noreply.github.com>
Co-authored-by: Jonathan Makunga <makung@amazon.com>
Co-authored-by: Prateek M Desai <prateekmdesai04@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-154.us-west-2.compute.internal>
Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com>
Co-authored-by: Samrudhi Sharma <154457034+samruds@users.noreply.github.com>
Co-authored-by: evakravi <69981223+evakravi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gwang111 gwang111 gwang111 approved these changes

@makungaj1 makungaj1 makungaj1 approved these changes

@mufaddal-rohawala mufaddal-rohawala mufaddal-rohawala approved these changes

@pintaoz-aws pintaoz-aws Awaiting requested review from pintaoz-aws pintaoz-aws is a code owner automatically assigned from aws/sagemaker-ml-frameworks

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@jiapinw @gwang111 @makungaj1 @mufaddal-rohawala