CodeVetter · Repo Unpack · Testing signals
ytdl-org/youtube-dl
Testing is unittest-based, executed by nosetests (pynose on Python >=3.9). Coverage splits into offline unit tests (utils, aes, jsinterp, swfinterp, compat, options, YoutubeDL internals, URL matching) and a large live-download suite generated dynamically from each extractor's embedded _TEST/_TESTS dictionaries. Network-dependent suites are explicitly excluded from 'offlinetest'/'core' runs. CI on GitHub Actions spans CPython 2.6-3.12, PyPy and Jython on Ubuntu and Windows, plus a flake8 lint job; the download test-set is continue-on-error.
- Test runner is nosetests: Makefile `test` runs `nosetests --verbose test` then flake8; devscripts/run_tests.sh selects core vs download sets via -I regex on test module names; tox.ini also drives nosetests with the same exclusion list.
- Extractor test cases are embedded in extractor classes as _TEST/_TESTS dicts; InfoExtractor.get_testcases yields them, test/helper.py's gettestcases aggregates across gen_extractors(), and test_download.py dynamically synthesizes one TestDownload.test_<Name> method per case.
- Download tests hit the real sites: they call ydl.extract_info on the case URL, retry up to 3 times on network-flavored errors (URLError, socket.timeout, HTTP 503), assert expected info_dict fields via expect_info_dict, verify the downloaded file exists and meets file_minsize, and optionally check an md5.
- test/parameters.json supplies base params for tests (test=true limiting downloads to _TEST_FILE_SIZE, writeinfojson=true, socket_timeout=20, fixup=never); test/helper.py merges an optional untracked local_parameters.json for credentials.
- Offline-only coverage is defined by exclusion: Makefile offlinetest and tox defaults exclude test_age_restriction, test_download, test_iqiyi_sdk_interpreter, test_socks, test_subtitles, test_write_annotations, test_youtube_lists and test_youtube_signature — i.e. anything needing network or live sites.
- Unit-level suites cover utils (test_utils.py), JS interpreter (test_jsinterp.py), AES (test_aes.py against testdata/), SWF interpreter (test_swfinterp.py against test/swftests/), URL matching across extractors (test_all_urls.py), YoutubeDL internals (test_YoutubeDL.py), options parsing (test_options.py), netrc, cache, traversal and compat.
- Test doubles exist in helper.py: FakeYDL (a YoutubeDL with auto_init=False that records download() calls and raises on trouble), FakeLogger, and expect_warnings/expect_info_dict assertion helpers that also fail when the extractor produces checkable fields missing from the test definition.
- CI (.github/workflows/ci.yml) builds a matrix over CPython 2.6 through 3.12 plus pypy and jython, ubuntu-22.04 primary with windows-2022 and jython extras; it includes custom steps to build Python 2.6/2.7/3.next and install nose/pynose per version; download and jython legs are continue-on-error.
- Lint is flake8 with repo-tuned ignores (E402,E501,E731,E741,W503) and excludes generated/excluded files; it runs as a separate CI job on Python 3.9 and as `make codetest`.
- Notable gaps: there is no coverage tooling enabled by default (commented-out --with-coverage), most of the ~800 extractors rely solely on their embedded live tests (marked skip/_WORKING when broken), and legacy interpreters (Python 2.6, Jython) mean modern test runners can't even execute parts of the matrix. (inferred)