AttributeError: 'NoneType' object has no attribute 'setdefault' on environment proxies

[tarmath]

AttributeError: 'NoneType' object has no attribute 'setdefault' on environment proxies

It appears there is a small bug in ‎Session.merge_environment_settings where the proxies variable can be None but setdefault is called on it anyway when env_proxies are present, causing an AttributeError to be thrown:

https://github.com/psf/requests/blob/main/src/requests/sessions.py#L759-L762

[sigmavirus24]

How are you encountering this? On which versions? Why did you skip the issue template?

[tarmath]

Hello! Apologies for not using the template above. I have included it below.

Additionally, I have unearthed more context while working on reproduction steps.

It appears that although the type hint signature in typeshed defines the proxies parameter as optional:

https://github.com/python/typeshed/blob/main/stubs/requests/requests/sessions.pyi#L307

In practice requests fallbacks to an empty dictionary which prevents hitting the issue:

https://github.com/psf/requests/blob/main/src/requests/sessions.py#L577

Expected Result

No Exception to be thrown.

Actual Result

AttributeError: 'NoneType' object has no attribute 'setdefault'

Reproduction Steps

Invoke this code with an environment that has a variable that ends with _PROXY, example:

HELLO_PROXY=abc python script.py

from requests import Request, Session

request = Request(method="GET", url="<https://disney.com")>

session = Session()
prepared_request = session.prepare_request(request)
settings = session.merge_environment_settings(prepared_request.url, None, None, None, None)
response = session.send(prepared_request, **settings)

System Information

{
 "chardet": {
 "version": null
 },
 "charset_normalizer": {
 "version": "3.3.2"
 },
 "cryptography": {
 "version": ""
 },
 "idna": {
 "version": "3.6"
 },
 "implementation": {
 "name": "CPython",
 "version": "3.9.6"
 },
 "platform": {
 "release": "23.2.0",
 "system": "Darwin"
 },
 "pyOpenSSL": {
 "openssl_version": "",
 "version": null
 },
 "requests": {
 "version": "2.31.0"
 },
 "system_ssl": {
 "version": "20000000"
 },
 "urllib3": {
 "version": "1.26.18"
 },
 "using_charset_normalizer": true,
 "using_pyopenssl": false
}

[nateprewitt]

I'd looked at this when the ticket first opened and the typing is due to ambiguity created by our top-level API defaults and the code inside merge_environment_settings.

We imply proxies can be None and even have some code written around that assumption. I'd personally rather be explicit that we require a Mapping and only a Mapping for this API since it doesn't really work without that. The only downside is a minor performance hit in the case we were previously skipping the merge in merge_settings. A sufficiently large proxies dictionary may see some performance impact, but I'm not sure that's going to be a noticeable case.