PyprojectTomlConfigSettingsSource or extend TomlConfigSettingsSource

[ITProKyle]

I have a need to accept user provided settings from either a file or environment variable. I personally like to limit the number of root-level files in my projects so I would like to provide the ability for my users to do that as well by enabling the use of a [tool.*] table for specifying settings. Looking through the current implementation of TomlConfigSettingsSource, there does not appear to be an easy way to do this.

I would like to propose one of two approaches to add this functionality as I am likely not the only one that would benefit from it:

1. Extend TomlConfigSettingsSource (and it's configuration) to enable specifying a table within the TOML file to load data from.
2. Create PyprojectTomlConfigSettingsSource (from TomlConfigSettingsSource) as it's own source with the functionality described above. Additionally, it could default to using the pyproject.toml file in the current working directory (or parent directory if not found) by default, if the source is enabled. Rational behind trying the parent directory relates to another tool I use, poetry. However, it will go beyond just the direct parent.

A case could also be made for both to be implemented - extending TomlConfigSettingsSource so that the table selection is more widely available and PyprojectTomlConfigSettingsSource for some file discovery be default.

---

I have a PoC of PyprojectTomlConfigSettingsSource that I am currently using. I'm open to contributing this if desired and the approach I took the way the maintainers would like to go. I'm just not starting there to conserve time.

from __future__ import annotations

from pathlib import Path
from typing import TYPE_CHECKING

from pydantic_settings import (
    BaseSettings,
    TomlConfigSettingsSource,
)
from pydantic_settings import (
    SettingsConfigDict as _SettingsConfigDict,
)

if TYPE_CHECKING:
    from pydantic_settings import PydanticBaseSettingsSource


class SettingsConfigDict(_SettingsConfigDict, total=False):
    """Overrides ``SettingsConfigDict`` to add ``pyproject.toml`` settings."""

    pyproject_toml_path: Path | str
    """Explicit path to a ``pyproject.toml`` file.

    If not provided, will attempt to load the ``pyproject.toml`` file located in the
    current working directory. If not found, the parent directory will be tried.
    Only one level higher will be tried.

    """

    toml_table_path: tuple[str, ...]
    """Path to the table to load.

    .. rubric:: Example
    .. code-block:: toml
        :caption: pyproject.toml

        [tool.poetry]

    .. code-block:: python

        SettingsConfigDict(toml_table_path=("tool", "poetry"))

    """


class PyprojectTomlConfigSettingsSource(TomlConfigSettingsSource):
    """``pyproject.toml`` config settings source."""

    def __init__(
        self,
        settings_cls: type[BaseSettings],
        toml_file: Path | None = None,
    ) -> None:
        """Instantiate class."""
        self.table_path: tuple[str, ...] = settings_cls.model_config.get(
            "toml_table_path", ()
        )
        self.toml_file_path = self._pick_pyproject_toml_file(
            toml_file or settings_cls.model_config.get("pyproject_toml_path")
        )
        self.toml_data = self._read_files(self.toml_file_path)
        for key in self.table_path:
            self.toml_data = self.toml_data.get(key, {})
        super(TomlConfigSettingsSource, self).__init__(settings_cls, self.toml_data)

    @staticmethod
    def _pick_pyproject_toml_file(provided: Path | None) -> Path:
        """Pick a pyproject.toml file path to use."""
        if provided:
            return provided.resolve()
        rv = Path.cwd() / "pyproject.toml"
        if not rv.is_file():
            rv = rv.parent.parent / "pyproject.toml"
        return rv


class Settings(BaseSettings):
    """Settings."""

    model_config = SettingsConfigDict(toml_table_path=("tool", "poetry"))

    name: str | None = None

    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,  # noqa: ARG003
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        """Example override to enable ``PyprojectTomlConfigSettingsSource``."""
        return (
            env_settings,
            dotenv_settings,
            PyprojectTomlConfigSettingsSource(settings_cls),
            init_settings,
        )


if __name__ == "__main__":
    print(Settings.model_validate({}))

[hramezani]

Thanks @ITProKyle for this feature request.

I think the second approach(creating a new setting source PyprojectTomlConfigSettingsSource) makes more sense.

Please remember to add documentation for the new setting source if you want to work on a PR