question about integration of existing parser

[braindevices]

The example in documentation does not really show how can I access the namespace the root parser:

import sys
from argparse import ArgumentParser

from pydantic_settings import BaseSettings, CliApp, CliSettingsSource

parser = ArgumentParser()
parser.add_argument('--food', choices=['pear', 'kiwi', 'lime'])


class Settings(BaseSettings):
    name: str = 'Bob'


# Set existing `parser` as the `root_parser` object for the user defined settings source
cli_settings = CliSettingsSource(Settings, root_parser=parser)

# Parse and load CLI settings from the command line into the settings source.
sys.argv = ['example.py', '--food', 'kiwi', '--name', 'waldo']
s = CliApp.run(Settings, cli_settings_source=cli_settings)
print(s.model_dump())
#> {'name': 'waldo'}

How can I access the value of food here?

[hramezani]

@kschwab Could you please take a look?

[kschwab]

Hi @braindevices, I think your case aligns more with the second example given in the docs:

import sys
from argparse import ArgumentParser

from pydantic_settings import BaseSettings, CliApp, CliSettingsSource

parser = ArgumentParser()
parser.add_argument('--food', choices=['pear', 'kiwi', 'lime'])


class Settings(BaseSettings):
    name: str = 'Bob'


# Set existing `parser` as the `root_parser` object for the user defined settings source
cli_settings = CliSettingsSource(Settings, root_parser=parser)

# Load CLI settings from pre-parsed arguments. i.e., the parsing occurs elsewhere and we
# just need to load the pre-parsed args into the settings source.
parsed_args = parser.parse_args(['--food', 'kiwi', '--name', 'ralph'])
s = CliApp.run(Settings, cli_args=parsed_args, cli_settings_source=cli_settings)
print(s.model_dump())
#> {'name': 'ralph'}

# To access food value
print(parsed_args.food)
#> kiwi

# Or using the root_parser attribute on the cli_settings object
print(cli_settings.root_parser.parse_args(['--food', 'kiwi', '--name', 'ralph']).food)
#> kiwi

In order to access food, for an external argparse parser, it has to parse the CLI to create the namespace. i.e., from the argparse docs:

The ArgumentParser.parse_args() method runs the parser and places the extracted data in a argparse.Namespace object

[braindevices]

Thanks. So if we use the run() directly we cannot get the extra args then. Then I am a bit confused about the 1st case. Because I thought the integration of an existing parser is only useful when we want to get the extra args. If not what is the 1st case good for?

[kschwab]

The 1st use case can be used to expand existing parsers without breaking compatibility. A good example of this would be extending the pytest CLI. There was a hand-wavy discussion of this in #391 (comment). Note, this was prior to the CliApp API, but the overall concept is the same.

What is the use case or desired flow you have in mind? Maybe we can massage it some to improve the experience.

[braindevices]

@kschwab thanks for the explanation. The 2nd example can do what I need. I was just curious what the 1st example is good for.

The pytest config hook actually interesting. But I still don't get why the 1st example get involved here.
Because, in the case of integration we have 2 cases:

A. we are controlling the entry point, so we are the one calling the parse method.
B. Other code controls the entry point, we do NOT call the parse method.

The pytest actually condition B, which is pytest control the entry point, we actually try to ask their parser to give us more info. Thus the 2nd example is preferred, so we do not call the parse method twice (I do not know if this is true.):

_cli_settings: CliSettingsSource
def pytest_addoption(parser):
    global _cli_settings
    _cli_settings = CliSettingsSource(
        Settings,
        root_parser=parser,
        parse_args_method=pytest.Parser.parse,
        add_argument_method=pytest.Parser.addoption,
        add_argument_group_method=pytest.Parser.getgroup,
        add_parser_method=None,
        add_subparsers_method=None,
        formatter_class=None,
    )

conf: Settings
def pytest_configure(config):
    global conf
    conf = CliApp.run(Settings, cli_args=config.option, cli_settings_source=cli_settings)

In my mind, 1st example should be used for the case A. So we can actually decide when to parse it, thus we can call CliApp.run(), and let the run() call parse method, then we get parameters we want, the others' code depends on the argparser is supposed to get what they want without any extra effort.

But of course now we do following still as 2nd example, because if we call parse method through run() the other code won't be able to get the parsed args at all:

from other import parser
from other import main_func
parsed_args = parser.parse_args()
s = CliApp.run(Settings, cli_args=parsed_args, cli_settings_source=cli_settings)
...
main_func(parsed_args)
...

unless other's code wrap the parser into some object like:

#### other module's code
class Wrapper:
...
    def parse(self):
        self.ns = self.parser.parse_args()
        return self.ns # which is rarely happen, if one use wrapper to record the state, then why would one return the state too.

wrapper = Wrapper(...)
...
do things with wrapper.ns

#### our code
from other import wrapper
cli_settings = CliSettingsSource(Settings, root_parser=wrapper, parse_args_method=wrapper.parse, ...)
s = CliApp.run(Settings, cli_settings_source=cli_settings)

So basically the only use case I can find is that the parse command is a some kind instance method which register the parsed args inside the parser instance and also return the namespace during the call. Is this correct?

[kschwab]

Sorry, you're right, the pytest example was more tailored towards use case 2. The main point for use case 1 is independent parsing from an external parser while sharing the same CLI. i.e. Case B is really:

Other code controls the entry point, and we may or may not call the parser method independently.

Because the parsers are integrated, they can operate simultaneously on the CLI args without interference.

Lets modify the example to help better demonstrate. Assuming the same Settings class setup etc., in conftest.py you could do:

import pytest
from settings import Settings
from pydantic_settings import CliSettingsSource

CLI_SETTINGS: CliSettingsSource
def pytest_addoption(parser):
    global CLI_SETTINGS
    CLI_SETTINGS = CliSettingsSource(
        Settings,
        root_parser=parser,
        parse_args_method=pytest.Parser.parse,
        add_argument_method=pytest.Parser.addoption,
        add_argument_group_method=pytest.Parser.getgroup,
        add_parser_method=None,
        add_subparsers_method=None,
        formatter_class=None,
    )

Then in your your actual tests you could do:

from pydantic_settings import CliApp
from conftest import CLI_SETTINGS
from settings import Settings

def test_my_settings():
    settings = CliApp.run(Settings, cli_settings_source=CLI_SETTINGS)

Where, both the pytest parser and settings parser can operate simultaneously, using a unified interface, without breaking or interfering with each other.

We actually employ use case 1 internally (not for pytest) to accomplish something similar. I think there is still room for improvement, but overall that's the main idea for use case 1.

[hramezani]

@kschwab do we need to do any action here? otherwise I want to close it

[kschwab]

@hramezani no, it is ok to close.