case_sensitive doesn't work in optional nested Settings models

[slingshotvfx]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

When using nested pydantic-settings models, I am seeing unexpected behavior with the case_sensitive flag when reading from a .env file.

When case_sensitive=True, things work as expected and are case sensitive, even though the docs say that on windows things should always be case insensitive.

However, when case_sensitive=False, I get ValidationErrors for Optional nested models regardless of what case I use.

Example Code

# .env
NOT_NESTED=works
NESTED__A=fails
nested__B=2

from pydantic import BaseModel
from pydantic_settings import BaseSettings, SettingsConfigDict

class NestedSettings(BaseModel):
    A: str
    B: int

class Settings(BaseSettings):
    not_nested: str
    nested: NestedSettings | None = None

    model_config = SettingsConfigDict(
        case_sensitive=False,
        env_file=".env",
        env_nested_delimiter="__",
        extra="forbid",
    )


print(Settings())

pydantic_core._pydantic_core.ValidationError: 2 validation errors for Settings
nested.A
  Field required [type=missing, input_value={'a': 'fails', 'b': '2'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.7/v/missing
nested.B
  Field required [type=missing, input_value={'a': 'fails', 'b': '2'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.7/v/missing

Changing case_sensitive=True and re-casing the .env file (e.g. nested__A=works) solves the issue. Removing the | None = None solves the issue too, although then of course the nested setting are required.

Python, Pydantic & OS Version

pydantic version: 2.7.1
        pydantic-core version: 2.18.2
          pydantic-core build: profile=release pgo=true
               python version: 3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)]
                     platform: Windows-11-10.0.22631-SP0
             related packages: fastapi-0.110.3 pydantic-settings-2.2.1 typing_extensions-4.11.0

[sydney-runkle]

@hramezani,

Could you please take a look at this? Thanks!

[hramezani]

Thanks @slingshotvfx for reporting this bug. I will investigate and try to prepare a fix