Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initialization of default JumpStart SageMaker Session is slow #4468

Open
CallumAtCarter opened this issue Mar 1, 2024 · 1 comment
Open
Labels

Comments

@CallumAtCarter
Copy link

Through the following import chain, the DEFAULT_JUMPSTART_SAGEMAKER_SESSION constant is initialized:

<module> (.venv\Lib\site-packages\sagemaker\jumpstart\constants.py:237)
<module> (.venv\Lib\site-packages\sagemaker\image_uris.py:24)
<module> (.venv\Lib\site-packages\sagemaker\estimator.py:30)
<module> (.venv\Lib\site-packages\sagemaker\__init__.py:18)

This involves creating many boto clients (at least 5 by my count), each time trying to find AWS credentials using botocore's CredentialResolver.load_credentials which (because I don't have credentials in environment variables, I'm using STS internally) ends up making slow network requests and is generally inefficient (searches through 11 credential providers). In total it is adding 13 seconds to my local startup time.

I'm using my own SageMaker Session whenever creating a new Predictor, and I don't believe I'm ever using the DEFAULT_JUMPSTART_SAGEMAKER_SESSION (if I modify the SDK source to skip its initialization everything still works). Could it be lazily loaded internally?

System information

  • SageMaker Python SDK version: 2.210.0
  • Python version: 3.11
@mohanasudhan
Copy link
Contributor

@evakravi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants