Incorrect AWS metadata server path when running on Fargate

[jonathanasdf]

Runs fine on EC2, but running on Fargate gives

    _gs.client = storage.Client()
  File "/usr/local/lib/python3.9/site-packages/google/cloud/storage/client.py", line 161, in __init__
    super(Client, self).__init__(
  File "/usr/local/lib/python3.9/site-packages/google/cloud/client.py", line 319, in __init__
    Client.__init__(
  File "/usr/local/lib/python3.9/site-packages/google/cloud/client.py", line 178, in __init__
    credentials, _ = google.auth.default(scopes=scopes)
  File "/usr/local/lib/python3.9/site-packages/google/auth/_default.py", line 473, in default
    project_id = credentials.get_project_id(request=request)
  File "/usr/local/lib/python3.9/site-packages/google/auth/external_account.py", line 269, in get_project_id
    self.before_request(request, "GET", url, headers)
  File "/usr/local/lib/python3.9/site-packages/google/auth/credentials.py", line 133, in before_request
    self.refresh(request)
  File "/usr/local/lib/python3.9/site-packages/google/auth/external_account.py", line 290, in refresh
    self._impersonated_credentials.refresh(request)
  File "/usr/local/lib/python3.9/site-packages/google/auth/impersonated_credentials.py", line 242, in refresh
    self._update_token(request)
  File "/usr/local/lib/python3.9/site-packages/google/auth/impersonated_credentials.py", line 255, in _update_token
    self._source_credentials.refresh(request)
  File "/usr/local/lib/python3.9/site-packages/google/auth/external_account.py", line 298, in refresh
    subject_token=self.retrieve_subject_token(request),
  File "/usr/local/lib/python3.9/site-packages/google/auth/aws.py", line 461, in retrieve_subject_token
    aws_security_credentials = self._get_security_credentials(request)
  File "/usr/local/lib/python3.9/site-packages/google/auth/aws.py", line 586, in _get_security_credentials
    role_name = self._get_metadata_role_name(request)
  File "/usr/local/lib/python3.9/site-packages/google/auth/aws.py", line 660, in _get_metadata_role_name
    response = request(url=self._security_credentials_url, method="GET")
  File "/usr/local/lib/python3.9/site-packages/google/auth/transport/requests.py", line 189, in __call__
    six.raise_from(new_exc, caught_exc)
  File "<string>", line 3, in raise_from
google.auth.exceptions.TransportError: HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/meta-data/iam/security-credentials (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f7839bd5280>, 'Connection to 169.254.169.254 timed out. (connect timeout=120)'))

Looking at https://stackoverflow.com/questions/57065458/cannot-access-instance-metadata-from-within-a-fargate-task it appears fargate has a different endpoint for iam metadata

[tomwphillips]

I encountered the same problem. As a workaround, I used boto3 to get credentials and then set the environment variables manually. The Google auth library will use the environment variables instead of querying the EC2 metadata service.

import os

import boto3
from google.auth import environment_vars


aws_credentials = boto3.Session().get_credentials().get_frozen_credentials()
os.environ[environment_vars.AWS_ACCESS_KEY_ID] = aws_credentials.access_key
os.environ[environment_vars.AWS_SECRET_ACCESS_KEY] = aws_credentials.secret_key
os.environ[environment_vars.AWS_SESSION_TOKEN] = aws_credentials.token

[ismailsimsek]

related googleapis/google-auth-library-java#957

[mateusz]

Same issue here - running from Fargate, using google-auth 2.23.3. Thanks @tomwphillips for the workaround - works well.

Please note in my case the reported error was OSError: [Errno 22] Invalid Argument