pytest-dev/pytest
github.com/pytest-dev/pytest ↗ · 705 files · 5.3 MB
Overview
pytest is a Python 3.10+ testing framework for application and library developers, exposed through the pytest CLI, python -m pytest, and pytest.main(). Its implementation organizes configuration, discovery, fixtures, execution, and reporting around plugin hooks, allowing built-in behavior and third-party extensions to share the same contracts. Distinctive capabilities include rewritten assertion diagnostics, scoped fixture dependency management, phase-specific reports, and persistent failure history. This brief examines commit 2887015cade4757385308e7a7d8083557fc637e2 through read-only source inspection; it does not establish runtime correctness.
System map
The runtime is a Python library and command-line test runner distributed as a package. Public APIs are collected in pytest, while most implementation lives in _pytest modules registered as plugins. Config, Session, fixture definitions, and report consumers own different stages of a run. Packaging, documentation hosting, release automation, and maintenance jobs form separate operational boundaries.
- Both pytest and py.test console scripts target _pytest.config:_console_main; python -m pytest calls that same function and raises SystemExit with its result.
- src/pytest/__init__.py exposes Config, Session, collectors, fixtures, assertion helpers, reports, and plugin decorators by importing their implementations from _pytest.
- get_config constructs a fresh PytestPluginManager and Config per invocation, processes plugin exclusions first, then imports default_plugins.
- wrap_session owns session configuration, startup, exception-to-exit-status handling, working-directory restoration, sessionfinish, and unconfiguration.
- Persistent state includes JSON Cache values, assertion-rewritten bytecode, and temporary directories; these have distinct formats and lifecycle owners.
- Packaging uses setuptools.build_meta and setuptools-scm, requires Python >=3.10, and declares pluggy, iniconfig, packaging, pygments, and conditional compatibility dependencies.
- tox defines wheel-based test environments, pre-commit linting, Sphinx documentation builds, and separate plugin integration checks; Read the Docs builds documentation with Python 3.14.
- Operational entrypoints include manual release-PR preparation and package deployment, weekly documentation link checking and plugin-list updates, and scheduled closure of needs-information issues.
Feature catalog
The primary user experience is selecting tests, discovering executable items, and receiving diagnostic reports. Fixtures and parametrization support reusable resources and generated test cases. Built-in integrations cover unittest, doctests, capture, logging, subtests, and machine-readable output. This catalog groups inspected capabilities into eight areas; it is bounded rather than an exhaustive listing of every CLI option.
- Users can run tests through pytest.main(args, plugins), while --collect-only skips execution after collection and collection errors stop the loop unless continuation is enabled.
- Python discovery selects .py modules using python_files unless explicitly selected, creates Class or Function nodes, and expands parametrized calls through pytest_generate_tests and Metafunc.
- -k evaluates case-insensitive substring matches across collected names and extra keywords; -m evaluates marker expressions, including matching marker keyword arguments.
- Fixtures support cached resource reuse, dependency-aware finalization, yield-based teardown, and monkeypatch operations that are undone after the requesting test or fixture finishes.
- Assertion rewriting compiles transformed test ASTs for detailed failures; public helpers include approx, raises, RaisesGroup, warns, and deprecated_call, while skip and xfail marks alter setup and reporting.
- Capture supports fd, sys, no, and tee-sys modes with -s disabling capture; caplog exposes messages, formatted text, LogRecord objects, tuples, and clearing.
- Built-in collection recognizes concrete unittest.TestCase subclasses and doctests from modules or text files; subtests.test provides individual reporting inside a test function.
- Run management includes last-failed filtering, failed-first ordering, cache inspection, and stepwise continuation; reporting includes terminal status output, JUnit XML, and opt-in pastebin publication.
Data flow
Arguments, environment options, configuration files, and plugins feed a mutable Config. Collection turns selected paths and node identifiers into Session items, and fixtures resolve those items' dependencies into function arguments. Runtime hooks produce reports containing outcomes and captured output. Report consumers render terminal output, serialize XML, and update cross-session failure state.
- Config.parse prepends PYTEST_ADDOPTS to explicit arguments, discovers configuration, then prepends configuration addopts, yielding configuration options followed by environment options and explicit arguments.
- Named configuration loaders are ordered from pytest.toml and .pytest.toml through INI, pyproject.toml, tox.ini, and setup.cfg; locate_config walks argument paths and their parents and returns the first valid configuration.
- Without explicit test arguments, _decide_args uses testpaths when invoked at rootpath, otherwise falls back to the invocation directory; pyargs changes testpaths handling from glob expansion to package arguments.
- Session.perform_collect resolves selection arguments, normalizes duplicates and overlaps unless keepduplicates is set, recursively expands collectors, then passes the item list through pytest_collection_modifyitems.
- Module.collect registers xunit setup fixtures and fixture factories; _genfunctions passes fixture information into Metafunc and produces one Function per callspec after pruning shadowed dependencies.
- TopRequest._fillfixtures stores resolved values in item.funcargs; pytest_pyfunc_call extracts only declared fixture argument names and calls the test with keyword arguments.
- CaptureManager attaches phase stdout and stderr sections to items; call_and_report wraps a phase in CallInfo, requests a TestReport, and sends it to logreport consumers such as TerminalReporter.
- LFPlugin converts cached strings to NodeId objects, updates failure membership from reports, and serializes changed history back to JSON; JUnit output separately writes an XML testsuites tree to the configured logfile.
Behavior traces
Three principal flows were followed end-to-end: startup into session dispatch, Python test collection and execution, and failure-history persistence into a later selection. Additional branches show assertion-cache loading, fixture teardown, XML output, and stepwise interruption. These traces describe default implementations; hook replacements and wrappers can change behavior. Error and worker conditions are retained where they affect execution order.
- Startup proceeds __main__ → _console_main → config._main → _prepareconfig → get_config → pytest_cmdline_parse; a version-only fast path returns before full infrastructure startup.
- Config.parse discovers settings → checks version → installs assertion rewriting → adjusts pythonpath → loads explicit plugins → optionally loads pytest11 entry points → loads PYTEST_PLUGINS → loads initial conftests.
- config._main dispatches pytest_cmdline_main → main.py wrap_session → configure → sessionstart → collection → runtestloop; finally it restores cwd, calls sessionfinish if startup completed, and unconfigures.
- Python collection follows pytest_collect_file → pytest_pycollect_makemodule → Module importtestmodule/import_path → Module.collect/parsefactories → _genfunctions → pytest_generate_tests → Function creation.
- A test executes pytest_runtest_protocol → runtestprotocol → setup report → call report only after successful setup → teardown report; Function.setup fills fixtures and Function.runtest dispatches pytest_pyfunc_call.
- Fixture execution resolves dependencies before checking cached results → finishes a mismatched parameter instance → registers dependency finalizers → calls pytest_fixture_setup → call_fixture_func; yield teardown advances the generator once more and rejects an additional yield.
- Failure persistence proceeds pytest_configure → Cache.for_config → LFPlugin cached-history load → report updates → sessionfinish changed-history write; a subsequent --lf run selects known failed items, while --ff places them first and workers skip saving history.
- Assertion imports call exec_module → _read_pyc →, on a miss, _rewrite_test and optional _write_pyc → exec; stepwise failure reports separately set Session.shouldstop and persist continuation state at sessionfinish outside workers.
Testing signals
Tests and CI were inspected, not executed; test existence is not a passing result. The repository tests pytest using pytest itself, including generated test projects through Pytester. Inspected cases exercise lifecycle reports, fixtures, assertion transforms, configuration, and cache boundaries. CI defines cross-platform and integration environments, but exclusions, disabled checks, and allowed failures limit what its configuration alone establishes.
- The default pytest configuration selects testing/, enables pytester, recognizes testing/python/*.py, and excludes example_scripts and plugins_integration from recursive discovery; these settings were inspected without running the suite.
- testing/conftest.py stabilizes terminal width and color environment variables and classifies Pytester subprocess or pexpect cases as slower tests.
- Runner tests inspect pass, failure, skip, setup failure, and teardown failure reports, including whether unsuccessful setup suppresses the call phase.
- Fixture regression tests assert that class-scoped setup and teardown occur once across multiple tests and that scoped setup exceptions are cached without repeating finalization.
- Assertion-rewrite helpers parse, transform, compile, and execute ASTs; configuration tests simulate pytest11 entry points and compare autoload disabling against explicit -p or PYTEST_PLUGINS enabling.
- Cache tests cover parent and absolute path escapes, normalization within the cache, invalid JSON fallback, and mocked open/write/close failures; these assertions do not establish coverage of every filesystem race.
- CI defines Windows, Ubuntu, macOS, CPython, PyPy, doctest, and plugin environments, installs built package artifacts through tox, and uploads coverage and JUnit reports; matrix.xfail jobs use continue-on-error.
- Coverage configuration enables branches, parallel data, and subprocess patching, while tox separates doctests and plugin integrations; its BDD wallet command is commented out as broken by an internal fixture-map change.
Risk map
The following are source observations and explicitly labeled hypotheses, not confirmed vulnerabilities. pytest intentionally imports and executes selected tests, plugins, and conftests in the invoking process. Filesystem boundaries include configurable cache directories, temporary roots, bytecode caches, and report destinations. Lifecycle and compatibility changes have a broad impact because plugins share internal execution and reporting hooks.
- Plugin autoload and explicit plugin imports execute installed or selected Python code; disabling pytest11 autoload still permits explicitly requested PYTEST_PLUGINS, and test modules are imported during collection.
- TempPathFactory.getbasetemp removes an existing explicitly supplied basetemp before recreating it; the default path lies under a system or PYTEST_DEBUG_TEMPROOT directory and includes ownership and symlink checks where UID information is available.
- Cache keys receive lexical containment validation, but _join_within does not resolve symlinks; hypothesis: protection against traversal through pre-existing filesystem links requires separate assessment. (inferred)
- Cache.set writes JSON directly to the destination, whereas cache-directory initialization and rewritten bytecode use rename/replace operations; hypothesis: concurrent independent cache writers can overwrite state or expose partial JSON, which readers then treat as a miss. (inferred)
- --pastebin=all uploads collected terminal output and --pastebin=failed uploads failure representations to https://bpa.st; create_new_paste calls urlopen without an explicit timeout and the feature emits a deprecation warning.
- Pytester.popen copies the parent environment, prepends the current directory to PYTHONPATH, and invokes subprocess.Popen; isolation changes must account for inherited state and child execution.
- JUnit sessionfinish creates parent directories and opens the configured logfile in write mode; assertion-cache loading unmarshals code and executes it after header and source-hash checks.
- Fixture-map changes already have a documented downstream compatibility impact: the BDD integration command is disabled, and fixture registration retains deprecated nodeid-based maps alongside preferred node-based registration.
Extension points
pytest exposes explicit plugin contracts for nearly every stage of its lifecycle. Extensions can register options, supply collectors, generate parameters, replace execution, and consume or transform reports. Fixtures and typed stashes provide resource and state extension mechanisms. Configuration loaders and the public re-export module are additional integration boundaries that require compatibility review.
- Installable plugins advertise a pytest11 entry point, while local conftests and PYTEST_PLUGINS supply additional plugin modules; -p supports explicit enabling and disabling except for essential plugins.
- pytest_addhooks adds hook specifications, and historic pytest_plugin_registered calls provide both prior registrations and subsequent registrations to participating plugins.
- pytest_addoption registers argparse options and configuration values through parser.addoption and parser.addini; implementations receive Config values through getoption/getini, and conftest support is limited to initial conftests.
- Custom collection can implement pytest_collect_file and return File/Item subclasses constructed with from_parent; the test harness contains a concrete YAML collector example.
- pytest_generate_tests receives Metafunc; module and class implementations are included through call_extra, and populated callspecs become individually named Function items.
- Fixture registration accepts scope, parameters, IDs, autouse, and node visibility; execution passes through pytest_fixture_setup and invokes pytest_fixture_post_finalizer after other fixture finalizers.
- Report plugins consume pytest_runtest_logreport or customize pytest_report_teststatus; TerminalReporter uses the returned category, letter, and word, while LogXML registers as a report consumer outside xdist workers.
- StashKey provides unique typed plugin-state keys on Config or Node stashes; configuration parsing has centralized named and suffix loader registries, and pytest/__init__.py controls public exposure of extension types.
Contributor handoff
Tests and CI were inspected, not executed, and the investigation is bounded rather than exhaustive. Future changes should begin with the public API, configuration lifecycle, and the specific plugin module involved. Lifecycle cleanup, configuration precedence, worker conditions, and downstream plugin compatibility are concrete review boundaries. Execution commands below are repository-documented follow-up checks for an authorized implementation session, not results from this investigation.
- Read pyproject.toml, tox.ini, pytest/__init__.py, and pytest/__main__.py first to establish package requirements, test commands, public exports, and the actual CLI entrypoint.
- For startup changes, read Config.parse and findpaths together: parsing stages combine options and load plugins, while loader ordering determines which configuration wins and which files are reported as ignored.
- For execution changes, preserve teardown after setup failure, request/funcargs cleanup in finally, sessionfinish gating, and final unconfiguration; these are separate cleanup obligations.
- For fixture changes, inspect dependency loading before cache checks and finish() invalidation; targeted tests include testing/python/fixtures.py cases for scoped reuse and cached setup exceptions.
- For cache or reporting changes, retain path containment and worker suppression, and inspect matching test assertions rather than assuming they passed; relevant targets include test_cacheprovider.py, test_runner.py, and test_terminal.py.
- CONTRIBUTING.rst documents tox -e py312 -- testing/test_config.py and direct pytest testing/test_config.py after development setup; tox -e linting runs configured pre-commit checks, including Ruff and mypy.
- Do not equate default suite inspection with integration coverage: plugin examples are excluded from discovery, tox runs integrations separately with BDD disabled, and selected CI matrix jobs allow failure.
- Compatibility traps include deprecated console_main, native rejection of unhandled async tests and fixtures, and deprecated nodeid fixture registration; documented behavior changes normally require an issue-numbered typed changelog fragment.
Languages
| Language | Files | Share of source |
|---|---|---|
| Python | 274 | 99% |
| YAML | 17 | docs/config |
| Markdown | 6 | docs/config |
| JSON | 2 | docs/config |
| TOML | 2 | 1% |
| CSS | 1 | docs/config |
| HTML | 1 | docs/config |
Start reading here
Probable source entrypoint: pyproject.toml — inferred from the directory tree and manifest, not the upstream docs.
Source roots
src/ (83 files)
Repository structure
| Top-level path | Files |
|---|---|
doc/ | 302 |
testing/ | 179 |
src/ | 83 |
changelog/ | 83 |
.github/ | 18 |
scripts/ | 10 |
bench/ | 7 |
extra/ | 1 |
Package
pyproject.toml — Miscellaneous internal changes vsrc/_pytest/_version.py, 0 declared dependencies.
CI & config entrypoints
.github/workflows/prepare-release-pr.yml— GitHub Actions workflow.github/workflows/test.yml— GitHub Actions workflow.github/workflows/stale.yml— GitHub Actions workflow.github/workflows/doc-check-links.yml— GitHub Actions workflow.github/workflows/deploy.yml— GitHub Actions workflow.github/workflows/update-plugin-list.yml— GitHub Actions workflow
Structural health signals
Average score 7.3/10 across 96 analyzed source files; 15 hotspots surfaced.
src/_pytest/fixtures.py— 1255 lines, score 4.7 (Large file, Deep nesting)testing/test_assertrewrite.py— 1564 lines, score 5.7 (Large file, Deep nesting)src/_pytest/assertion/rewrite.py— 1220 lines, score 5.7 (Large file, Deep nesting)
These are deterministic review leads — size, churn, and structural signals — not proof of a bug in this project.
Recent history
2887015cade42026-09-29 — [pre-commit.ci] pre-commit autoupdate (#15077)b1de564f337c2026-09-28 — build(deps): Bump pytest-rerunfailures in /testing/plugins_integration (#15110)24e7f30b020f2026-09-28 — build(deps): Bump codecov/codecov-action from 7.1.0 to 7.1.1 (#15111)c333125d977b2026-09-27 — [automated] Update plugin list (#15107)8721173580392026-09-24 — testing: ignore Cython's PyPy collection_type warning (#15096)