error: PythonPipBuilder:ResolveDependencies - {protobuf==3.16.0(wheel)}

[vithyavani-bit]

Description:

Steps to reproduce:

Observed result:

Expected result:

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS:
2. sam --version:
3. AWS region:

Add --debug flag to command you are running

[aahung]

@vithyavani-bit Can you elaborate more, especially steps to reproduce?

[joobcode]

Build previously succeeded in version 1.22.0, now failing:

Build is in a container "sam build --use-container"

Seeing same in same versions 1.22.0 and 1.23.0

also:

docker run -it --rm public.ecr.aws/sam/build-python3.8
bash-4.2# sam --version
SAM CLI, version 1.23.0
bash-4.2# exit

my project has the following requirements.txt:

boto3
google-api-python-client
google-auth

[joobcode]

❯ sam build --use-container --debug
2021-05-08 13:50:33,745 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2021-05-08 13:50:33,746 | Using config file: samconfig.toml, config environment: default
2021-05-08 13:50:33,746 | Expand command line arguments to:
2021-05-08 13:50:33,746 | --template_file=/Users/xxx/aws/zc-gcal/sam-z-cal-event-add/template.yaml --use_container --build_dir=.aws-sam/build --cache_dir=.aws-sam/cache
2021-05-08 13:50:33,816 | 'build' command is called
2021-05-08 13:50:33,816 | Starting Build inside a container
2021-05-08 13:50:33,827 | Collected default values for parameters: {'CredentialSSMPath': '/zoho_calendar/credentials.json'}
2021-05-08 13:50:33,868 | 3 stacks found in the template
2021-05-08 13:50:33,868 | Collected default values for parameters: {'CredentialSSMPath': '/zoho_calendar/credentials.json'}
2021-05-08 13:50:33,915 | 3 resources found in the stack
2021-05-08 13:50:33,915 | Collected default values for parameters: {'CredentialSSMPath': '/zoho_calendar/credentials.json'}
2021-05-08 13:50:33,950 | Found Serverless function with name='EventAddFunction' and CodeUri='code/'
2021-05-08 13:50:33,950 | --base-dir is not presented, adjusting uri code/ relative to /Users/xxx/aws/zc-gcal/sam-z-cal-event-add/template.yaml
2021-05-08 13:50:33,950 | Collected default values for parameters: {'CredentialSSMPath': '/zoho_calendar/credentials.json'}
2021-05-08 13:50:33,982 | Instantiating build definitions
2021-05-08 13:50:33,990 | Same function build definition found, adding function (Previous: BuildDefinition(python3.8, /Users/xxx/aws/zc-gcal/sam-z-cal-event-add/code, Zip, , d1643bd8-6cb9-4e02-9b41-159a7d90846e, {}, {}, []), Current: BuildDefinition(python3.8, /Users/xxx/aws/zc-gcal/sam-z-cal-event-add/code, Zip, , dfa246ac-3102-4be3-b105-b71267c91f84, {}, {}, []), Function: Function(name='EventAddFunction', functionname='EventAddFunction', runtime='python3.8', memory=None, timeout=60, handler='app.lambda_handler', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/Users/xxx/aws/zc-gcal/sam-z-cal-event-add/code', environment={'Variables': {'CRED_SSM_PATH': '/zoho_calendar/credentials.json'}}, rolearn='arn:aws:lambda:us-east-1:123456789012:function:EventAddRole', layers=[], events={'EventAdd': {'Type': 'HttpApi', 'Properties': {'Path': '/event/add', 'Method': 'POST', 'ApiId': 'HttpApi'}}}, metadata=None, inlinecode=None, codesign_config_arn=None, stack_path=''))
2021-05-08 13:50:33,992 | Building codeuri: /Users/xxx/aws/zc-gcal/sam-z-cal-event-add/code runtime: python3.8 metadata: {} functions: ['EventAddFunction']
2021-05-08 13:50:33,992 | Building to following folder /Users/xxx/aws/zc-gcal/sam-z-cal-event-add/.aws-sam/build/EventAddFunction

Fetching public.ecr.aws/sam/build-python3.8:latest Docker container image......
2021-05-08 13:50:40,612 | Mounting /Users/xxx/aws/zc-gcal/sam-z-cal-event-add/code as /tmp/samcli/source:ro,delegated inside runtime container
Using the request object from command line argument
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r /tmp/samcli/source/requirements.txt --dest /tmp/samcli/scratch
Full dependency closure: {pyparsing==2.4.7(wheel), requests==2.25.1(wheel), six==1.16.0(wheel), uritemplate==3.0.1(wheel), google-auth==1.30.0(wheel), pyasn1-modules==0.2.8(wheel), boto3==1.17.69(wheel), urllib3==1.26.4(wheel), chardet==4.0.0(wheel), botocore==1.20.69(wheel), rsa==4.7.2(wheel), setuptools==56.1.0(wheel), s3transfer==0.4.2(wheel), httplib2==0.19.1(wheel), protobuf==3.16.0(wheel), jmespath==0.10.0(wheel), google-api-core==1.26.3(wheel), cachetools==4.2.2(wheel), googleapis-common-protos==1.53.0(wheel), python-dateutil==2.8.1(wheel), idna==2.10(wheel), pytz==2021.1(wheel), pyasn1==0.4.8(wheel), certifi==2020.12.5(wheel), google-api-python-client==2.3.0(wheel), google-auth-httplib2==0.1.0(wheel), packaging==20.9(wheel)}
initial compatible: {pyparsing==2.4.7(wheel), requests==2.25.1(wheel), six==1.16.0(wheel), uritemplate==3.0.1(wheel), google-auth==1.30.0(wheel), pyasn1-modules==0.2.8(wheel), urllib3==1.26.4(wheel), boto3==1.17.69(wheel), chardet==4.0.0(wheel), botocore==1.20.69(wheel), rsa==4.7.2(wheel), setuptools==56.1.0(wheel), s3transfer==0.4.2(wheel), httplib2==0.19.1(wheel), jmespath==0.10.0(wheel), google-api-core==1.26.3(wheel), cachetools==4.2.2(wheel), googleapis-common-protos==1.53.0(wheel), python-dateutil==2.8.1(wheel), idna==2.10(wheel), pytz==2021.1(wheel), pyasn1==0.4.8(wheel), certifi==2020.12.5(wheel), google-api-python-client==2.3.0(wheel), google-auth-httplib2==0.1.0(wheel), packaging==20.9(wheel)}
initial incompatible: {protobuf==3.16.0(wheel)}
Downloading missing wheels: {protobuf==3.16.0(wheel)}
calling pip download --only-binary=:all: --no-deps --platform manylinux2014_x86_64 --implementation cp --abi cp38 --dest /tmp/samcli/scratch protobuf==3.16.0
compatible wheels after second download pass: {pyparsing==2.4.7(wheel), requests==2.25.1(wheel), six==1.16.0(wheel), uritemplate==3.0.1(wheel), google-auth==1.30.0(wheel), pyasn1-modules==0.2.8(wheel), boto3==1.17.69(wheel), urllib3==1.26.4(wheel), chardet==4.0.0(wheel), botocore==1.20.69(wheel), rsa==4.7.2(wheel), setuptools==56.1.0(wheel), s3transfer==0.4.2(wheel), httplib2==0.19.1(wheel), jmespath==0.10.0(wheel), google-api-core==1.26.3(wheel), cachetools==4.2.2(wheel), googleapis-common-protos==1.53.0(wheel), python-dateutil==2.8.1(wheel), idna==2.10(wheel), pytz==2021.1(wheel), pyasn1==0.4.8(wheel), certifi==2020.12.5(wheel), google-api-python-client==2.3.0(wheel), google-auth-httplib2==0.1.0(wheel), packaging==20.9(wheel)}
Build missing wheels from sdists (C compiling True): set()
compatible after building wheels (no C compiling): {pyparsing==2.4.7(wheel), requests==2.25.1(wheel), six==1.16.0(wheel), uritemplate==3.0.1(wheel), google-auth==1.30.0(wheel), pyasn1-modules==0.2.8(wheel), boto3==1.17.69(wheel), urllib3==1.26.4(wheel), chardet==4.0.0(wheel), botocore==1.20.69(wheel), rsa==4.7.2(wheel), setuptools==56.1.0(wheel), s3transfer==0.4.2(wheel), httplib2==0.19.1(wheel), jmespath==0.10.0(wheel), google-api-core==1.26.3(wheel), cachetools==4.2.2(wheel), googleapis-common-protos==1.53.0(wheel), python-dateutil==2.8.1(wheel), idna==2.10(wheel), pytz==2021.1(wheel), pyasn1==0.4.8(wheel), certifi==2020.12.5(wheel), google-api-python-client==2.3.0(wheel), google-auth-httplib2==0.1.0(wheel), packaging==20.9(wheel)}
Build missing wheels from sdists (C compiling False): set()
compatible after building wheels (C compiling): {pyparsing==2.4.7(wheel), requests==2.25.1(wheel), six==1.16.0(wheel), uritemplate==3.0.1(wheel), google-auth==1.30.0(wheel), pyasn1-modules==0.2.8(wheel), boto3==1.17.69(wheel), urllib3==1.26.4(wheel), chardet==4.0.0(wheel), botocore==1.20.69(wheel), rsa==4.7.2(wheel), setuptools==56.1.0(wheel), s3transfer==0.4.2(wheel), httplib2==0.19.1(wheel), jmespath==0.10.0(wheel), google-api-core==1.26.3(wheel), cachetools==4.2.2(wheel), googleapis-common-protos==1.53.0(wheel), python-dateutil==2.8.1(wheel), idna==2.10(wheel), pytz==2021.1(wheel), pyasn1==0.4.8(wheel), certifi==2020.12.5(wheel), google-api-python-client==2.3.0(wheel), google-auth-httplib2==0.1.0(wheel), packaging==20.9(wheel)}
Final compatible: {httplib2==0.19.1(wheel), pyparsing==2.4.7(wheel), jmespath==0.10.0(wheel), requests==2.25.1(wheel), google-api-core==1.26.3(wheel), six==1.16.0(wheel), cachetools==4.2.2(wheel), uritemplate==3.0.1(wheel), google-auth==1.30.0(wheel), googleapis-common-protos==1.53.0(wheel), python-dateutil==2.8.1(wheel), pyasn1-modules==0.2.8(wheel), idna==2.10(wheel), boto3==1.17.69(wheel), urllib3==1.26.4(wheel), pytz==2021.1(wheel), pyasn1==0.4.8(wheel), certifi==2020.12.5(wheel), google-api-python-client==2.3.0(wheel), google-auth-httplib2==0.1.0(wheel), chardet==4.0.0(wheel), botocore==1.20.69(wheel), rsa==4.7.2(wheel), setuptools==56.1.0(wheel), s3transfer==0.4.2(wheel), packaging==20.9(wheel)}
Final incompatible: {protobuf==3.16.0(wheel)}
Final missing wheels: {protobuf==3.16.0(wheel)}
PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 39, in execute
    package_builder.build_dependencies(
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 137, in build_dependencies
    self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 207, in build_site_packages
    raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {protobuf==3.16.0(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 269, in run
    action.execute()
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 45, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {protobuf==3.16.0(wheel)}
Builder workflow failed
Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 39, in execute
    package_builder.build_dependencies(
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 137, in build_dependencies
    self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 207, in build_site_packages
    raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {protobuf==3.16.0(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 269, in run
    action.execute()
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 45, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {protobuf==3.16.0(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/__main__.py", line 117, in main
    builder.build(
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/builder.py", line 125, in build
    return workflow.run()
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 76, in wrapper
    func(self, *args, **kwargs)
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 276, in run
    raise WorkflowFailedError(workflow_name=self.NAME, action_name=action.NAME, reason=str(ex))
aws_lambda_builders.exceptions.WorkflowFailedError: PythonPipBuilder:ResolveDependencies - {protobuf==3.16.0(wheel)}
2021-05-08 13:50:56,014 | Build inside container returned response {"jsonrpc": "2.0", "id": 1, "error": {"code": 400, "message": "PythonPipBuilder:ResolveDependencies - {protobuf==3.16.0(wheel)}"}}

Build Failed
2021-05-08 13:50:56,347 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '7adeef71-fc5b-4c3d-ade3-ae0fe0f389d1', 'installationId': '2e273f2e-b106-49c6-8516-d04504094d65', 'sessionId': '08e8bfdb-3ff4-4979-ac1d-d11063a1a5a0', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.10', 'samcliVersion': '1.23.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 22601, 'exitReason': 'BuildInsideContainerError', 'exitCode': 1}}]}
2021-05-08 13:50:57,103 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: PythonPipBuilder:ResolveDependencies - {protobuf==3.16.0(wheel)}

[idwise-team]

Same here, started getting the same exact error today.

[joobcode]

adding the following to my requirements.txt resolved this:

protobuf==3.15.0

Appears that protocol was upgraded on 7th May 2021, with the following change log entries specific to Python:

  Python
  * Fix some constness / char literal issues being found by MSVC standard conforming mode (#8344)
  * Switch on "new" buffer API (#8339)
  * Enable crosscompiling aarch64 python wheels under dockcross manylinux docker image (#8280)
  * Fixed a bug in text format where a trailing colon was printed for repeated field.
  * When TextFormat encounters a duplicate message map key, replace the current
    one instead of merging.

Seems related to change 8280 and manylinux2014 wheel

protocolbuffers/protobuf#8280

@jfuss this might be related to other issues you addressed?

[idwise-team]

Just realised that! thanks, all worked now.
protobuf==3.15.8 worked as well.

[vithyavani-bit]

Issue got resolved by adding protobuf==3.15.0 to requirement.txt Thank you so much@joobcode

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.