pydantic/pydantic-settings
github.com/pydantic/pydantic-settings ↗ · 55 files · 878 KB
pydantic-settings Settings management using Pydantic.
Overview
pydantic-settings is a Python library for developers who want application configuration expressed as typed Pydantic models. It combines constructor arguments, environment variables, dotenv files, secret files, optional configuration files, and cloud secret providers through an ordered source pipeline, then delegates validation to Pydantic. The same model definitions can generate command-line interfaces with subcommands, help, argument serialization, and synchronous or asynchronous command handlers.
System map
The runtime is an importable Python package whose primary boundary is model construction. BaseSettings orchestrates source selection and merging, while source modules own input acquisition and preparation. CLI execution adds parser and command-dispatch boundaries around the same settings machinery. Packaging, development checks, documentation rendering, and tagged releases are configured separately.
- The package initializer exposes BaseSettings, SettingsConfigDict, CliApp, source providers, CLI annotations, decoding markers, and version metadata as the public import surface.
- BaseSettings subclasses Pydantic BaseModel; its constructor prepares sources, merges their mappings, and passes the result to BaseModel.__init__ for validation.
- PydanticBaseSettingsSource defines the dictionary-producing source contract, field acquisition/preparation hooks, and access to prior aggregate and per-source state.
- Environment, dotenv, and flat secret sources are built into construction; JSON, TOML, YAML, pyproject, nested secrets, and cloud integrations are additional source classes selected through customization.
- Runtime retention includes a process-local settings cache keyed by exact model class and separate per-class construction locks; source mappings and merge state otherwise belong to settings resolution.
- Hatchling builds the distribution and reads its dynamic version from version.py; the manifest requires Python >=3.10, Pydantic >=2.7.0, python-dotenv, typing-extensions, and typing-inspection.
- Optional extras supply YAML, TOML, Azure, AWS, and GCP dependencies; Makefile development commands use uv for installation, Ruff lint/format checks, mypy, and pytest under coverage.
- Documentation uses strict MkDocs Material configuration; the inspected CI publishes a built package to PyPI only for tags after its aggregate check succeeds, using a release environment and OIDC permission.
Feature catalog
The user-facing surface is primarily Python APIs plus model-generated command-line behavior. Configuration features cover precedence, aliases, nested values, decoding, files, and secrets. Optional cloud sources adapt remote secret stores to model fields. Reloading, shared caching, diagnostic logging, help generation, and CLI serialization round out the operational APIs.
- SettingsConfigDict and constructor overrides configure case sensitivity, environment prefixes and prefix targets, empty-value filtering, nested delimiters/split limits, null strings, and enum parsing.
- Complex environment values support JSON decoding and nested-key overrides; NoDecode, ForceDecode, and enable_decoding control decoding, while nested_model_default_partial_update can contribute model/dataclass default mappings.
- Dotenv loading supports multiple files, encoding, bounded parent-directory discovery, and only_existing or match_prefix filtering; explicit _env_file=None disables loading.
- JSON, TOML, and YAML sources accept multiple paths or packaged Traversable resources and optional deep merging; TOML selects tuple-based tables, YAML selects dotted sections, and pyproject defaults to tool.pydantic-settings.
- Flat secrets load field-matched UTF-8 files from ordered directories; nested secrets additionally support directory-shaped nesting, separate prefix/case rules, missing-directory policies, and a default 16 MiB size limit.
- Cloud integrations include AWS Secrets Manager with optional VersionId, recursive decrypted AWS SSM parameters, Azure enabled-secret lookup and name conversion, and GCP project resolution plus per-field SecretVersion metadata.
- CLI models support subcommands, positional and variadic arguments, dual/toggle/explicit boolean flags, unknown-argument capture, suppression, shortcuts, kebab-case naming, mutually exclusive groups, and customizable parser methods.
- Operational APIs include settings_cached/settings_clear_cache, documented in-place __init__ reloading, PYDANTIC_SETTINGS_DEBUG source diagnostics, and CliApp run, run_subcommand, serialize, format_help, and print_help.
Data flow
Inputs become dictionaries before final Pydantic validation. Constructor inputs and field aliases are normalized so source values can merge under compatible keys. Environment-shaped providers share decoding and nested expansion, whereas structured files parse into mappings first. The pipeline retains intermediate source dictionaries for dependent sources and optional diagnostics, then produces a validated model.
- _settings_init_sources resolves explicit constructor controls against model_config, with ENV_FILE_SENTINEL preserving the distinction between omitted dotenv configuration and an explicit None.
- InitSettingsSource matches constructor keys using configured case rules and alias preference, writes matched values under canonical preferred aliases, removes duplicate matches, and retains unmatched extras.
- EnvSettingsSource snapshots os.environ during initialization; parse_env_vars normalizes names, optionally drops empty strings, and marks configured null strings with EnvNoneType.
- Field acquisition searches extracted environment names in order; preparation decodes complex values and deep-merges delimiter-expanded nested values over a decoded top-level mapping.
- Structured file sources call ConfigFileSourceMixin._read_files, skip non-files, merge later file mappings over earlier ones shallowly or deeply, then feed the result through InitSettingsSource.
- CLI parsing produces a namespace or dictionary, resolves repeated values and selected subcommands, marks unselected subcommands as null, and converts the result into env_vars for inherited environment-style preparation.
- Before each source call, _settings_build_values supplies current_state and settings_sources_data; it records the source result and applies deep_update(source_state, state), giving previously merged sources priority.
- After merging, unchanged default-source values are removed, optional debug output is emitted, original constructor key names are restored, and BaseModel.__init__ validates the resulting mapping.
Behavior traces
The inspected flows cover ordinary construction, CLI command execution, file-backed secrets, cloud-backed secrets, caching, and serialization. Their execution order explains when I/O happens and which state a source can observe. File and cloud paths load configuration into models; the shown flows do not persist model changes back to their input stores. Cache reuse retains validated instances within the process.
- Construction flow: BaseSettings.__init__ → _settings_init_sources → built-in source constructors → settings_customise_sources → _settings_build_values → BaseModel.__init__; sources normally run init, environment, dotenv, flat secrets, defaults.
- Nested environment flow: EnvSettingsSource._load_env_vars → parse_env_vars → inherited __call__ → get_field_value → prepare_field_value → decode_complex_value/explode_env_vars; nested entries override overlapping decoded JSON keys before model validation.
- Dotenv flow: DotEnvSettingsSource initialization invokes its _load_env_vars override → _read_env_files → _resolve_config_file → dotenv_values → parse_env_vars; later files replace earlier keys, then __call__ applies field extraction and extra-variable filtering.
- Flat-secret flow: SecretsSettingsSource.__call__ expands configured directories, warns for missing paths, rejects existing non-directories, then inherited field traversal calls get_field_value → find_case_path → UTF-8 read_text().strip(); later directories are searched first.
- CLI flow: CliApp.run checks the model type → _settings_init_sources creates/connects CliSettingsSource → arguments load into env_vars → sources merge and validate → _run_cli_cmd invokes the root handler if present; a handler calling run_subcommand selects the annotated child and requires its handler.
- GCP flow: source construction initially returns an empty environment mapping; the merge loop supplies current_state → source.__call__ → _load_env_vars → _resolve_gcp_project → GoogleSecretManagerMapping → field lookup → access_secret_version, using explicit project, prior-source project, then auth defaults.
- Cache flow: settings_cached checks the guarded class cache, obtains a per-class lock, double-checks the entry, constructs cls() outside the global guard, and stores it; settings_clear_cache removes only the exact class entry and cannot revoke held or in-flight references.
- Serialization flow: CliApp.serialize builds a CliSettingsSource → _serialized_args walks model fields and parser metadata → _serialize_value converts enums/containers → _flatten_serialized_args orders optional/positional segments, subcommands, and finally verbatim unknown arguments.
Testing signals
Tests and CI were inspected, not executed, so this report contains no passing-test or measured-coverage result. Pytest tests cover precedence, aliases, CLI execution and round trips, caching, files, and mocked cloud behavior. CI specifies a cross-platform Python matrix and combined coverage gates. Inspection is bounded, and mocked providers, platform skips, and coverage exclusions limit what those checks establish.
- Pytest discovers tests under tests, treats warnings as errors with explicit exceptions, and is run by make test through uv run coverage run -m pytest --durations=10; these commands were inspected, not executed.
- test_precedence_and_merging.py asserts init > environment > dotenv > secrets > defaults, nested merge preservation, and custom environment-first ordering with AliasChoices/AliasGenerator.
- Cache tests assert identity reuse, exact-class invalidation, single construction across eight threads, cross-thread nested loading, concurrent unrelated classes, collection after clearing, and retry after failed validation.
- CLI tests exercise subcommand dispatch and alternate handlers, async handlers with and without a running loop, worker-thread exception propagation, and serialize/parse round trips for list and dictionary styles.
- Nested-secret regressions assert external directory/file symlinks do not populate settings; JSON tests distinguish shallow/deep multi-file merging, and fixtures simulate non-UTF-8 default decoding.
- Inspected AWS tests use moto and optional-dependency skips, while Azure and GCP tests mock SDK operations; these fixtures establish intended local behavior without demonstrating live IAM, endpoint, or service compatibility.
- Documentation examples use pytest-examples for linting and execution expectations, but skip unsupported platforms/implementations, honor per-example skip/version requirements, and can mark expected failures.
- Inspected CI specifies Python 3.10–3.14 across Linux/macOS/Windows, repeats tests after uninstalling PyYAML, combines branch coverage, and requires 98% overall/100% library coverage; configured exclusions and matrix aggregation prevent interpreting these gates as exhaustive single-platform verification.
Risk map
The sensitive boundaries are configuration acquisition, secret retrieval, diagnostic output, shared caches, and command invocation. Findings below are source observations or explicitly labeled hypotheses, not confirmed vulnerabilities. Several paths intentionally read outside the repository or communicate through cloud SDKs. Tests document some protections and failure policies, but this inspection supplies no runtime security verification.
- PYDANTIC_SETTINGS_DEBUG plus DEBUG logging emits reprs of complete source dictionaries before model validation; the implementation and documentation explicitly warn that environment, dotenv, and secret values can appear.
- Configured file paths can expand home directories or point outside the repository; dotenv discovery can search cwd parents and accepts FIFOs, making path trust and potentially blocking reads application concerns.
- Flat SecretsSettingsSource searches directory entries then uses is_file/read_text without the resolved-root confinement implemented by NestedSecretsSettingsSource; the two APIs therefore provide different symlink-handling boundaries.
- Nested secrets share a confined, cycle-guarded iterator between size accounting and loading, but those operations occur separately; a concurrently mutable secret tree could change between validation and reading, a source-derived race hypothesis requiring runtime investigation. (inferred)
- GCP value acquisition catches broad exceptions and returns None for many access failures; tests explicitly expect PermissionDenied to become None, allowing lower-priority sources/defaults or final missing-field validation to determine the result.
- AWS SDK calls accept caller-selected endpoint URLs, SSM requests recursive decrypted parameters, Azure takes a supplied credential, and GCP can invoke google.auth.default; these are network and authentication boundaries delegated to optional SDKs.
- settings_cached returns a shared mutable object per class/process, retains old values until clearing, and does not invalidate existing references; even frozen models can contain mutable collections shared across callers.
- Core and CLI code import private Pydantic internals; settings I/O is synchronous, and async CLI handlers invoked inside an existing loop run in a worker thread followed by join, so the public call remains blocking.
Extension points
Customization centers on explicit source composition rather than automatic plugin discovery. Sources can reuse field preparation and observe higher-priority state. CLI extensions use model metadata and an argparse-compatible adapter surface. File readers, cloud-client injection, validation hooks, and public exports provide additional concrete places to extend behavior.
- Override settings_customise_sources to add, remove, or reorder dictionary-producing sources; its tuple establishes highest-first priority, and the orchestrator appends DefaultSettingsSource afterward.
- Subclass PydanticBaseSettingsSource and implement get_field_value/__call__; inherited prepare_field_value, field_is_complex, and decode_complex_value supply reusable conversion hooks.
- Dependent sources can read current_state and settings_sources_data populated immediately before __call__; GCP project resolution is an implemented example of consuming prior-source state.
- New structured-file providers can combine ConfigFileSourceMixin with InitSettingsSource and implement _read_file, retaining shared multi-file ordering, optional deep merging, and alias normalization.
- CliSettingsSource accepts an external root_parser and injectable parse/add/group/subparser/help methods; adapters must honor argparse-compatible signatures, and external parsers cannot request case-insensitive matching.
- CLI Annotated helpers and CliMutuallyExclusiveGroup extend model-driven parser generation; _CliArg and _parser_map connect field metadata to argument destinations, subcommand models, and serialization.
- CliApp accepts an alternate cli_cmd_method_name, while Pydantic validators can transform or check the model after settings acquisition; inspected tests exercise alternate command dispatch and an after-model validator.
- GCP supports prebuilt secret_client injection and SecretVersion field metadata; adding public APIs or provider dependencies also involves the explicit package exports and optional-extras manifest.
Contributor handoff
Begin changes at the source boundary responsible for the behavior and read its matching tests. Precedence, aliases, decoding, source-construction timing, and shared-state lifetime are the main cross-module contracts. Tests and CI were inspected, not executed; the suggested commands are future checks requiring execution authorization. This brief is bounded source inspection and supplies no passing results or confirmed vulnerabilities.
- Read main.py source orchestration, sources/base.py contracts, and providers/env.py preparation before modifying settings resolution; changes there affect every environment-derived provider and final model construction.
- Preserve highest-first source order and deep_update(source_state, state); use precedence and alias regressions to assess changes rather than assuming later sources overwrite earlier ones.
- Distinguish configuring a file source from enabling it: setting json_file/toml_file/yaml_file alone does not insert the source, and _settings_warn_unused_config_keys warns when the corresponding source is absent.
- Preserve ENV_FILE_SENTINEL versus None, platform-specific environment case handling, and explicit UTF-8 reads; these encode observable configuration and portability contracts.
- For CLI changes, inspect main.py and providers/cli.py together with dispatch and round-trip tests; root handlers are optional in code, child handlers are required, and unknown serialized arguments must remain last.
- Keep source initialization timing explicit: environment loading occurs in its constructor, AWS reads during inherited initialization, and GCP deliberately waits for __call__ so previous sources can provide its project.
- Tests and CI were inspected, not executed. When authorized, start with focused pytest cases in the affected test file, then use repository make lint, make mypy, and make test; CI's combined coverage gates are configuration evidence rather than current results.
- Maintain Ruff's Python 3.10 target, 120-character lines and single-quote formatting, strict mypy checks, and shared InitState warning deduplication; unresolved field definitions direct users to model_rebuild rather than silently assuming annotations are complete.
Languages
| Language | Files | Share of source |
|---|---|---|
| Python | 37 | 97% |
| YAML | 6 | docs/config |
| CSS | 2 | docs/config |
| Markdown | 2 | docs/config |
| HTML | 1 | docs/config |
| JSON | 1 | docs/config |
| TOML | 1 | 3% |
Start reading here
Probable source entrypoint: pyproject.toml — inferred from the directory tree and manifest, not the upstream docs.
Source roots
pydantic_settings/ (23 files)
Source layout — pydantic_settings/
| Path | Files |
|---|---|
pydantic_settings/sources/ | 17 |
pydantic_settings/__init__.py | 1 |
pydantic_settings/exceptions.py | 1 |
pydantic_settings/main.py | 1 |
pydantic_settings/py.typed | 1 |
pydantic_settings/utils.py | 1 |
pydantic_settings/version.py | 1 |
Repository structure
| Top-level path | Files |
|---|---|
pydantic_settings/ | 23 |
tests/ | 16 |
docs/ | 5 |
.github/ | 3 |
Docs found: README.mddocs/index.md
Package
pyproject.toml — pydantic-settings, 8 declared dependencies.
yamltomlazure-key-vaultaws-secrets-manageraws-systems-managergcp-secret-managergoogle-cloud-secret-manager>=2.23.1"]
CI & config entrypoints
README.md— Repository readme.github/workflows/ci.yml— GitHub Actions workflow
Structural health signals
Average score 7.7/10 across 20 analyzed source files; 3 hotspots surfaced.
pydantic_settings/sources/providers/cli.py— 1067 lines, score 5.7 (Large file, Deep nesting)pydantic_settings/main.py— 1011 lines, score 5.7 (Large file, Deep nesting)pydantic_settings/sources/base.py— 644 lines, score 6.2 (Large file, Deep nesting)
These are deterministic review leads — size, churn, and structural signals — not proof of a bug in this project.
Recent history
5927b441848c2026-10-02 — Fix empty JSON arrays in CLI list arguments (#994)a6d68139a18a2026-10-02 — Honor `secrets_case_sensitive` on Windows by overriding `_load_env_vars` (#1001)6daf93d844ac2026-10-02 — Fix `CliApp.serialize` for models with `CliUnknownArgs` fields (#992)aabb8698424c2026-10-02 — build(deps): bump the python-packages group with 3 updates (#999)d41a298d3fa32026-10-01 — build(deps): bump astral-sh/setup-uv from 10.0.1 to 10.2.0 in the github-actions group (#998)