Validation error for 3 levels of nested dicts in v2.3.0

[bpicardat]

Hello,
My model does not work anymore with the latest version of pydantic-settings.
Here is a test that reproduces my issue (env is the fixture from the pydantic-settings tests):

def test_nested_dicts(env):
    class Settings(BaseSettings):
        nested: Dict[str, Dict[str, Dict[str, str]]]

        model_config = SettingsConfigDict(env_nested_delimiter='__')

    env.set('nested__foo__a__b', 'bar')
    s = Settings()
    assert s.model_dump() == {'nested': {'foo': {'a': {'b': 'bar'}}}}

This test passes in version 2.2.1 of pydantic-settings but fails in version 2.3.0 with error:

E               pydantic_settings.sources.SettingsError: error parsing value for field "nested" from source "EnvSettingsSource"

.venv/lib/python3.11/site-packages/pydantic_settings/sources.py:377: SettingsError

This happens with Python 3.11.9 on Linux, and the following packages:

Package           Version
----------------- -------
annotated-types   0.7.0
iniconfig         2.0.0
packaging         24.0
pip               24.0
pluggy            1.5.0
pydantic          2.7.3
pydantic_core     2.18.4
pydantic-settings 2.3.0
pytest            8.2.2
python-dotenv     1.0.1
setuptools        69.5.1
typing_extensions 4.12.1
wheel             0.43.0

Is this considered a regression or should I find an alternative way to solve my issue?

[xaniasd]

chiming in on this if I may, the following fails with the same error when I set nested__test__val=myval. I think it doesn't have to do with the number of nesting levels, but generally setting values in dicts. I suspect
these lines, i.e. it's trying to parse a non-JSON value as such. This snippet does work with pydantic-settings < 2.3.0

from pydantic import BaseModel
from pydantic_settings import BaseSettings, SettingsConfigDict

class NestedModel(BaseModel):
    val: str

class Settings(BaseSettings):
    val: str
    nested: dict[str, NestedModel]
    model_config = SettingsConfigDict(
        env_nested_delimiter="__",
    )


if __name__ == "__main__":
    Settings(val="test")

[hramezani]

Thanks @bpicardat for reporting this. Yes, this is a regression in a853a13

I will prepare a fix for that. meanwhile you can fix the problem by setting env like:

env.set('nested', '{"foo": {"a": {"b": "bar"}}}')

[hramezani]

Thanks @xaniasd for reporting. you can also set the env like nested='{"test": {"val": "myval"}}' meanwhile

I am trying to prepare a fix

[hramezani]

@bpicardat @xaniasd I created #301 to fix the problem. Could you please confirm? then I can prepare a patch release soon

[bpicardat]

@hramezani I confirm that the branch issue-300 fixed my issue. Thank you !

[hramezani]

Fixed in ad07a57

[hramezani]

The fix has been released in pydantic-settings 2.3.1