CodeVetter · Repo Unpack · Contributor handoff
ytdl-org/youtube-dl
For an agent working in this repo: read CONTRIBUTING.md's developer instructions and extractor/common.py's giant docstring first; the info-dict contract and the 'first suitable() wins' ordering are the two invariants that matter most. Python 2.6-compatible style (unicode_literals, compat_* aliases, no f-strings, %-formatting) is mandatory and enforced by CI. Tests are nosetests; extractor changes get a live _TEST; run flake8 with the repo's setup.cfg ignores. Generated files (lazy_extractors.py, README.md, completions, man page) are produced by Makefile/devscripts — edit sources, not artifacts.
- Read first: youtube_dl/extractor/common.py's InfoExtractor docstring (the entire info-dict schema), CONTRIBUTING.md 'DEVELOPER INSTRUCTIONS' + 'youtube-dl coding conventions', youtube_dl/YoutubeDL.py's options docstring, and youtube_dl/__init__.py _real_main for how params map to CLI flags.
- Adding an extractor: new file in youtube_dl/extractor/, class name must end in 'IE', _VALID_URL with a named (?P<id>...) group is conventional, import the class in extractors.py (order = precedence), include a _TEST dict with a real URL; commit style is '[name] Add new extractor'.
- Mandatory info-dict fields are id and title plus either url or formats; everything else must be extracted defensively (helpers: _search_regex with default=, try_get, traverse_obj, int_or_none/float_or_none) because extractors must tolerate missing optional data.
- Run tests with `python test/test_download.py TestDownload.test_<Name>` for a single extractor; `make offlinetest` (or nosetests with the exclusion list) for network-free checks; full `make test` requires live network access and will fail on sites that changed since the pinned commit.
- Lint with `flake8 .` honoring setup.cfg (ignores E402/E501/E731/E741/W503, excludes generated files); style is %-formatting, `from __future__ import unicode_literals` at top of every file, compat_* imports from youtube_dl/compat.py instead of py3-only stdlib paths.
- Lazy-extractor constraint is a known trap: if you override suitable() or reference _VALID_URL oddly, devscripts/make_lazy_extractors.py output can break — keep suitable() self-importing and prefer plain _VALID_URL regexes; regenerate via `make lazy-extractors` or `python setup.py build_lazy_extractors`.
- GenericIE must remain the last resort: _ALL_CLASSES appends it explicitly and many extractors emit url_result(ie=GenericIE.ie_key()) delegates; do not reorder extractors casually — the first regex match wins.
- Generated artifacts to not hand-edit: youtube_dl/extractor/lazy_extractors.py (devscripts/make_lazy_extractors.py), README.md/CONTRIBUTING.md (devscripts/make_readme.py/make_contributing.py), docs/supportedsites.md (make_supportedsites.py), completions and man page (Makefile targets).
- Safe edit boundaries: adding a new extractor file + extractors.py import is low-blast-radius; editing extractor/common.py, utils.py, YoutubeDL.py or downloader/common.py affects every site — keep changes additive, preserve py2.6 compatibility, and remember `_real_main` mutates the global std_headers dict and params dicts are shared references. (inferred)
- Debugging support built in: --verbose prints the debug header (version, git HEAD, Python, exe versions, proxy map), --write-pages/--dump-pages dump raw server responses, --print-traffic logs HTTP, and --youtube-print-sig-code prints the extracted signature function — use these before touching extractor code.