# Testing signals — yt-dlp/yt-dlp

> Canonical page: https://codevetter.com/unpack/yt-dlp/testing-signals

# Testing signals — yt-dlp/yt-dlp

Scanned commit c7fb478d21e9e59524befbe23f7801bb267fb880 (2026-09-16). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Tests live in `test/` (pytest + unittest hybrid). Two tiers: 'core' tests (utils, networking, jsinterp, postprocessors, options) run offline under `-m 'not download'`, and 'download' tests that hit live sites, generated dynamically from each extractor's `_TESTS`/`_WEBPAGE_TESTS`. CI runs core tests across CPython 3.10-3.15 and PyPy on Linux+Windows with reruns; lint is ruff + autopep8. Coverage is explicitly unimplemented. The extractor download tests depend on live third-party sites and are therefore inherently flaky and mostly excluded from core CI.
- pytest is the test runner (`pytest~=9.0`, `pytest-rerunfailures`), configured with `-ra`, `--tb=short`, `strict_markers`, and a custom `download` marker. — [pyproject.toml#L112-115](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/pyproject.toml#L112-L115) [pyproject.toml#L473-483](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/pyproject.toml#L473-L483)
- `test_download.py` dynamically generates `TestDownload` unittest methods from `gettestcases()`/`getwebpagetestcases()`, which iterate every extractor's `_TESTS`/`_WEBPAGE_TESTS`; tests download real URLs and verify info dicts plus md5 checksums. — [test/test_download.py#L61-100](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/test_download.py#L61-L100) [test/helper.py#L88-97](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/helper.py#L88-L97)
- `devscripts/run_tests.py` is the canonical entrypoint: `core` => `pytest -m 'not download'`, `download` => `-m download`, and bare names map to `test/test_download.py::TestDownload::test_<IE>`; it falls back to `python -m unittest` when pytest is absent. — [devscripts/run_tests.py#L39-82](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/devscripts/run_tests.py#L39-L82)
- Core CI (`core.yml`) runs on ubuntu+windows across py3.11-3.15 and pypy-3.11 (py3.10 on Windows + quick-test.yml on Linux), skipping `yt_dlp/extractor/**` changes except youtube/common/extractors, and adds `--flaky` only when networking files changed. — [.github/workflows/core.yml#L1-105](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/.github/workflows/core.yml#L1-L105)
- `test_all_urls.py` asserts URL->extractor matching precedence by calling `ie.suitable(url)` across all extractors, guarding against `_VALID_URL` regressions like youtu.be and playlist/tab disambiguation. — [test/test_all_urls.py#L17-60](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/test_all_urls.py#L17-L60)
- Networking tests are parametrized per request handler via a `handler` pytest fixture that looks up classes in `_REQUEST_HANDLERS` and supports `skip_handler`/`skip_handler_if` markers for missing optional deps. — [test/conftest.py#L1-40](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/conftest.py#L1-L40)
- Test parameters come from `test/parameters.json` (`test: true`, `writeinfojson`, `format: 'b/bv'`) mergeable with a git-ignored `local_parameters.json` for secrets/overrides. — [test/parameters.json](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/parameters.json) [test/helper.py#L21-33](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/helper.py#L21-L33)
- Dedicated suites exist for the YouTube PO-token provider framework (`test_pot_framework.py`, director, builtin providers) and the JS-challenge/ejs integration (`test_ejs_integration.py`). — [test/test_pot/test_pot_framework.py](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/test_pot/test_pot_framework.py) [test/test_jsc/test_ejs_integration.py](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/test_jsc/test_ejs_integration.py)
- `make test` = `pytest -Werror` then `make codetest` (ruff check + autopep8 --diff); `make offlinetest` runs `-m 'not download'` plus lint; hatch env `hatch-test` adds pytest-randomly and pytest-xdist. — [Makefile#L79-88](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/Makefile#L79-L88) [pyproject.toml#L223-238](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/pyproject.toml#L223-L238)
- Code coverage is not wired up: the hatch `run-cov` script is a stub that echoes 'Code coverage not implemented' and exits 1. — [pyproject.toml#L238](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/pyproject.toml#L238)
- Download tests self-skip for extractors marked `_WORKING = False` and for `expected_exception` cases, acknowledging the live-site flakiness; warnings are escalated to errors via a `report_warning` override. — [test/test_download.py#L95-100](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/test_download.py#L95-L100) [test/test_download.py#L41-53](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/test_download.py#L41-L53)
- A `FakeYDL` helper subclasses `YoutubeDL` with `auto_init=False`, `trouble`->raise, and `download`->record, so unit tests exercise the real orchestrator without network. — [test/helper.py#L60-75](https://github.com/yt-dlp/yt-dlp/blob/c7fb478d21e9e59524befbe23f7801bb267fb880/test/helper.py#L60-L75)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)