Sagemaker Local Mode with Local Code setting and HuggingFaceModel still requires S3 access

[jaketothepast]

Describe the bug

When deploying a HuggingFace model with model data on disk, sagemaker SDK still tries to access the AWS API to determine the Sagemaker default bucket. I don't currently have access to my AWS credentials, so this is failing with access denied errors. I would like to be able to run locally without needing S3 access, and this is blocking me from doing so.

To reproduce

In an environment without AWS Access (no credentials setup).

from sagemaker.huggingface import HuggingFaceModel
from sagemaker import Session, get_execution_role

from sagemaker.local import LocalSession

# sagemaker_session = LocalSession()
# sagemaker_session.config = {'local': {'local_code': True}}
huggingface_model = HuggingFaceModel(
   model_data="file:///path/to/model.tar.gz",  # path to your trained SageMaker model
   role='SageMakerRole',                                            # IAM role with permissions to create an endpoint
   transformers_version="4.26",                           # Transformers version used
   pytorch_version="1.13",                                # PyTorch version used
   py_version='py39',
)

huggingface_model.deploy(
    initial_instance_count=1,
    instance_type='local'
)

Maybe I still need access to AWS via the HF inference toolkit pulling the correct ECR image? However, that doesn't seem to be where it's failing.

Expected behavior
Local mode to deploy an endpoint into a Docker container.

System information
A description of your system. Please provide:

[tool.poetry.dependencies]
python = "^3.12"
sagemaker = {extras = ["local"], version = "^2.217.0"}
jupyterlab = "^4.1.8"
setuptools = "^69.5.1"

• CPU or GPU: CPU
• Custom Docker image (Y/N): N

Additional context

It looks like regardless of the local code setting in ~/.sagemaker/config.yaml, the SDK will still try to access the S3 API to determine the default bucket name for sagemaker.

From

sagemaker-python-sdk/src/sagemaker/model.py

Line 694 in b17d332

bucket, key_prefix = s3.determine_bucket_and_prefix(

.

    def _upload_code(self, key_prefix: str, repack: bool = False) -> None:
        """Uploads code to S3 to be used with script mode with SageMaker inference.

        Args:
            key_prefix (str): The S3 key associated with the ``code_location`` parameter of the
                ``Model`` class.
            repack (bool): Optional. Set to ``True`` to indicate that the source code and model
                artifact should be repackaged into a new S3 object. (default: False).
        """
        local_code = utils.get_config_value("local.local_code", self.sagemaker_session.config)

        bucket, key_prefix = s3.determine_bucket_and_prefix(
            bucket=self.bucket,
            key_prefix=key_prefix,
            sagemaker_session=self.sagemaker_session,
        )

        if (self.sagemaker_session.local_mode and local_code) or self.entry_point is None:
            self.uploaded_code = None
        elif not repack:
            self.uploaded_code = fw_utils.tar_and_upload_dir(
                session=self.sagemaker_session.boto_session,
                bucket=bucket,
                s3_key_prefix=key_prefix,
                script=self.entry_point,
                directory=self.source_dir,
                dependencies=self.dependencies,
                kms_key=self.model_kms_key,
                settings=self.sagemaker_session.settings,
            )

      # ...

Within determine_bucket_and_prefix , sagemaker_session.default_bucket() is called. In my environment, this still fails even with a LocalSession object.

from sagemaker.huggingface import HuggingFaceModel
from sagemaker import Session, get_execution_role

from sagemaker.local import LocalSession

sagemaker_session = LocalSession()
sagemaker_session.config = {'local': {'local_code': True}}
sagemaker_session.default_bucket()

---------------------------------------------------------------------------
ClientError                               Traceback (most recent call last)
[/Users/jacob.windle/Projects/sagemaker_local_testing/TestM2MLocalMode.ipynb](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/TestM2MLocalMode.ipynb) Cell 2 line 8
      [6](vscode-notebook-cell:/Users/jacob.windle/Projects/sagemaker_local_testing/TestM2MLocalMode.ipynb#W1sZmlsZQ%3D%3D?line=5) sagemaker_session = LocalSession()
      [7](vscode-notebook-cell:/Users/jacob.windle/Projects/sagemaker_local_testing/TestM2MLocalMode.ipynb#W1sZmlsZQ%3D%3D?line=6) sagemaker_session.config = {'local': {'local_code': True}}
----> [8](vscode-notebook-cell:/Users/jacob.windle/Projects/sagemaker_local_testing/TestM2MLocalMode.ipynb#W1sZmlsZQ%3D%3D?line=7) sagemaker_session.default_bucket()

File [~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:586](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:586), in Session.default_bucket(self)
    [584](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:584) default_bucket = self._default_bucket_name_override
    [585](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:585) if not default_bucket:
--> [586](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:586)     default_bucket = generate_default_sagemaker_bucket_name(self.boto_session)
    [587](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:587)     self._default_bucket_set_by_sdk = True
    [589](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:589) self._create_s3_bucket_if_it_does_not_exist(
    [590](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:590)     bucket_name=default_bucket,
    [591](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:591)     region=region,
    [592](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:592) )

File [~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:7359](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:7359), in generate_default_sagemaker_bucket_name(boto_session)
   [7351](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:7351) """Generates a name for the default sagemaker S3 bucket.
   [7352](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:7352) 
   [7353](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:7353) Args:
   [7354](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:7354)     boto_session (boto3.session.Session): The underlying Boto3 session which AWS service
   [7355](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:7355) """
   [7356](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:7356) region = boto_session.region_name
   [7357](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/sagemaker/session.py:7357) account = boto_session.client(
...
-> [1021](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/botocore/client.py:1021)     raise error_class(parsed_response, operation_name)
   [1022](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/botocore/client.py:1022) else:
   [1023](https://file+.vscode-resource.vscode-cdn.net/Users/jacob.windle/Projects/sagemaker_local_testing/~/Projects/sagemaker_local_testing/.venv/lib/python3.12/site-packages/botocore/client.py:1023)     return parsed_response

ClientError: An error occurred (InvalidClientTokenId) when calling the GetCallerIdentity operation: The security token included in the request is invalid.

Why is a LocalSession still trying to access S3?

[jaketothepast]

If this is actually a problem, and not just my own ignorance, I'm happy to contribute.