Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to change AWS_PROFILE for Resource in between S3Path calls #161

Open
nathanpowellcma opened this issue Feb 22, 2024 · 1 comment · May be fixed by #162
Open

Add ability to change AWS_PROFILE for Resource in between S3Path calls #161

nathanpowellcma opened this issue Feb 22, 2024 · 1 comment · May be fixed by #162

Comments

@nathanpowellcma
Copy link

Hi,

Thanks for the great package - finding it super useful! We use AWS_PROFILEs and sometimes need to switch between profiles to access different buckets. We spotted the _cleanup() method in the tests but can't use it because the way the resource is created doesn't use the Session and therefore will always use the profile at the time that boto3 is imported (or, at least, this is my understanding). The following code snippet works if you change default_resource to return boto3.Session().resource('s3').

# %%
from s3path import accessor, S3Path
import os

# %%
def _cleanup():
    accessor.configuration_map.get_configuration.cache_clear()
    accessor.configuration_map.get_general_options.cache_clear()
    accessor.configuration_map.is_setup = False
    
# %%
os.environ["AWS_PROFILE"] = "shared1"
test_path = S3Path("/data1/test.txt")
test_path.read_text()

# %%
_cleanup()

# %%
os.environ["AWS_PROFILE"] = "shared2"
test_path1 = S3Path("/data2/test.txt")
test_path1.write_text()

I've made a PR which passes the tests. I'm assuming that you wouldn't want to provide some sort of cache clearing / cleanup utility to the users, but I'm happy to implement it myself given that it's relatively easy.

@nathanpowellcma nathanpowellcma linked a pull request Feb 22, 2024 that will close this issue
@liormizr
Copy link
Owner

Hi @nathanpowellcma I added a comment in your PR
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants