Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws4 tests fails because of "UnboundLocalError: local variable 'region_name' referenced before assignment" #520

Open
nadavMiz opened this issue Jun 27, 2023 · 0 comments

Comments

@nadavMiz
Copy link
Contributor

when running aws4 tests with S3_USE_SIGV4=true and host = 'localhost' causes the test to fail with the following error:

bash-4.4$ S3_USE_SIGV4=true S3TEST_CONF=test_ceph_s3_config.conf tox s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4
GLOB sdist-make: /root/node_modules/noobaa-core/src/test/system_tests/ceph_s3_tests/s3-tests/setup.py
WARNING: could not copy distfile to //.tox/distshare
py inst-nodeps: /root/node_modules/noobaa-core/src/test/system_tests/ceph_s3_tests/s3-tests/.tox/dist/s3tests-0.0.1.zip
py installed: WARNING: The directory '/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.,attrs==22.2.0,boto==2.49.0,boto3==1.23.10,botocore==1.26.10,certifi==2023.5.7,charset-normalizer==2.0.12,distlib==0.3.6,filelock==3.4.1,gevent==22.10.2,greenlet==2.0.2,httplib2==0.22.0,idna==3.4,importlib-metadata==4.8.3,importlib-resources==5.4.0,iniconfig==1.1.1,isodate==0.6.1,jmespath==0.10.0,lxml==4.9.2,munch==3.0.0,packaging==21.3,platformdirs==2.4.0,pluggy==1.0.0,py==1.11.0,pyparsing==3.1.0,pytest==7.0.1,python-dateutil==2.8.2,pytz==2023.3,PyYAML==6.0,requests==2.27.1,s3tests @ file:///root/node_modules/noobaa-core/src/test/system_tests/ceph_s3_tests/s3-tests/.tox/dist/s3tests-0.0.1.zip,s3transfer==0.5.2,six==1.16.0,toml==0.10.2,tomli==1.2.3,tox==3.28.0,typing_extensions==4.1.1,urllib3==1.26.16,virtualenv==20.17.1,zipp==3.6.0,zope.event==4.6,zope.interface==5.5.2
py run-test-pre: PYTHONHASHSEED='1650885128'
py runtests: commands[0] | pytest s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.6.8, pytest-7.0.1, pluggy-1.0.0
cachedir: .tox/py/.pytest_cache
rootdir: /root/node_modules/noobaa-core/src/test/system_tests/ceph_s3_tests/s3-tests, configfile: pytest.ini
collected 1 item                                                                                                                                                                                           

s3tests/functional/test_headers.py E                                                                                                                                                                 [100%]

================================================================================================== ERRORS ==================================================================================================
___________________________________________________________________ ERROR at setup of test_bucket_create_bad_amz_date_before_epoch_aws4 ____________________________________________________________________

    @pytest.fixture(scope="package")
    def configfile():
>       setup()

s3tests/functional/__init__.py:376: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
s3tests/functional/__init__.py:367: in setup
    nuke_prefixed_buckets(prefix=prefix)
s3tests/functional/__init__.py:121: in nuke_prefixed_buckets
    nuke_prefixed_buckets_on_conn(prefix, name, conn)
s3tests/functional/__init__.py:67: in nuke_prefixed_buckets_on_conn
    for bucket in conn.get_all_buckets():
.tox/py/lib/python3.6/site-packages/boto/s3/connection.py:443: in get_all_buckets
    response = self.make_request('GET', headers=headers)
.tox/py/lib/python3.6/site-packages/boto/s3/connection.py:671: in make_request
    retry_handler=retry_handler
.tox/py/lib/python3.6/site-packages/boto/connection.py:1071: in make_request
    retry_handler=retry_handler)
.tox/py/lib/python3.6/site-packages/boto/connection.py:927: in _mexe
    request.authorize(connection=self)
.tox/py/lib/python3.6/site-packages/boto/connection.py:377: in authorize
    connection._auth_handler.add_auth(self, **kwargs)
.tox/py/lib/python3.6/site-packages/boto/auth.py:755: in add_auth
    **kwargs)
.tox/py/lib/python3.6/site-packages/boto/auth.py:574: in add_auth
    string_to_sign = self.string_to_sign(req, canonical_request)
.tox/py/lib/python3.6/site-packages/boto/auth.py:514: in string_to_sign
    sts.append(self.credential_scope(http_request))
.tox/py/lib/python3.6/site-packages/boto/auth.py:496: in credential_scope
    region_name = self.determine_region_name(http_request.host)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <boto.auth.S3HmacAuthV4Handler object at 0x7f55fb338a20>, host = 'localhost'

    def determine_region_name(self, host):
        # S3's different format(s) of representing region/service from the
        # rest of AWS makes this hurt too.
        #
        # Possible domain formats:
        # - s3.amazonaws.com (Classic)
        # - s3-us-west-2.amazonaws.com (Specific region)
        # - bukkit.s3.amazonaws.com (Vhosted Classic)
        # - bukkit.s3-ap-northeast-1.amazonaws.com (Vhosted specific region)
        # - s3.cn-north-1.amazonaws.com.cn - (Beijing region)
        # - bukkit.s3.cn-north-1.amazonaws.com.cn - (Vhosted Beijing region)
        parts = self.split_host_parts(host)
    
        if self.region_name is not None:
            region_name = self.region_name
        else:
            # Classic URLs - s3-us-west-2.amazonaws.com
            if len(parts) == 3:
                region_name = self.clean_region_name(parts[0])
    
                # Special-case for Classic.
                if region_name == 's3':
                    region_name = 'us-east-1'
            else:
                # Iterate over the parts in reverse order.
                for offset, part in enumerate(reversed(parts)):
                    part = part.lower()
    
                    # Look for the first thing starting with 's3'.
                    # Until there's a ``.s3`` TLD, we should be OK. :P
                    if part == 's3':
                        # If it's by itself, the region is the previous part.
                        region_name = parts[-offset]
    
                        # Unless it's Vhosted classic
                        if region_name == 'amazonaws':
                            region_name = 'us-east-1'
    
                        break
                    elif part.startswith('s3-'):
                        region_name = self.clean_region_name(part)
                        break
    
>       return region_name
E       UnboundLocalError: local variable 'region_name' referenced before assignment

.tox/py/lib/python3.6/site-packages/boto/auth.py:690: UnboundLocalError
------------------------------------------------------------------------------------------ Captured stdout setup -------------------------------------------------------------------------------------------
Deleting buckets on main
Cleaning buckets from connection main prefix 'ceph-flereorziy1omk1x0axzfy7a-'.
============================================================================================= warnings summary =============================================================================================
.tox/py/lib/python3.6/site-packages/boto/plugin.py:40
  /root/node_modules/noobaa-core/src/test/system_tests/ceph_s3_tests/s3-tests/.tox/py/lib/python3.6/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================================= short test summary info ==========================================================================================
ERROR s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4 - UnboundLocalError: local variable 'region_name' referenced before assignment
======================================================================================= 1 warning, 1 error in 1.39s ========================================================================================
ERROR: InvocationError for command '/root/node_modules/noobaa-core/src/test/system_tests/ceph_s3_tests/s3-tests/.tox/py/bin/pytest s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4' (exited with code 1)
_________________________________________________________________________________________________ summary __________________________________________________________________________________________________
ERROR:   py: commands failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant