CodeVetter · Repo Unpack · Testing signals
curl/curl
curl's testing is unusually broad and almost entirely in-tree. The centre of gravity is `tests/data/` — 2,119 XML-ish test-case files run by `tests/runtests.pl` against protocol servers curl itself ships (sws, socksd, tftpd, mqttd, rtspd, dnsd, plus Perl/Python FTP/TFTP/RTSP/DICT/telnet servers). Around that sit 80 C unit tests for library internals, 6 `tunit` tests for tool-only code, 273 `libtest` programs exercising the public API, and 31 pytest modules that test HTTP/2, HTTP/3, proxying, SSL and WebSockets against real nghttpx/caddy/vsftpd. Each run can be repeated in torture mode (fail every allocation in turn), under valgrind, or event-based. Coverage gaps are structural rather than accidental: platform-specific code (Windows/VMS/AmigaOS/MSDOS branches) and the rarer TLS/QUIC backend combinations can only be exercised by the matching CI job.
- The primary suite is 2,119 declarative test files in tests/data/, each an XML-like document with `<info><keywords>`, a `<reply>` server script, a `<client><command>` invocation, and a `<verify><protocol>` expectation — plus optional `<limits>` on allocation counts.
- `tests/runtests.pl` is the harness; its documented flags cover torture testing (`-t`), valgrind control, event-based runs, and keyword/feature selection such as `FTPS` or `!feat:proxy`, with per-test result flags (s/r/p/d/u/P/o/e/m/v/E) reported per run.
- Protocol servers are part of the repo, so tests need no external services: tests/server/ holds sws.c (HTTP), socksd.c, tftpd.c, mqttd.c, rtspd.c, dnsd.c and sockfilt.c, complemented by ftpserver.pl, http2-server.pl, http3-server.pl, sshserver.pl and dictserver.py.
- 80 unit tests under tests/unit/ exercise library internals directly (llist, hash, splay, strparse, urlapi, cookie parsing …) using the `UNITTEST_BEGIN_SIMPLE` macro and a `test_unitNNNN(const char *arg)` entrypoint.
- Tool-only code has its own unit directory, tests/tunit/, with 6 files named after the test number they back (tool1394, tool1604, tool1621, tool1622, tool1623, tool1720) — this is where `src/` helpers get direct coverage.
- 273 `libtest/libNNNN.c` programs plus a set of `cli_*.c` drivers (cli_h2_pausing, cli_hx_download, cli_ws_pingpong, cli_tls_session_reuse, cli_ev_download) test the public easy/multi/WebSocket APIs as real linked programs.
- The pytest suite in tests/http/ has 31 modules numbered by area — 01 basic, 02 download, 07 upload, 09 push, 10/13 proxy and proxy auth, 17 ssl use, 19 shutdown, 20 websockets, 22 HTTPSRR, 30-32 vsftpd, 40 socks, 50/51 scp+sftp, 60 h3 proxy — driven by `conftest.py` and a `testenv` package.
- Make targets select the mode: `tests/Makefile.am` maps `test`/`quiet-test`/`am-test`/`ci-test`/`full-test`/`nonflaky-test`/`torture-test`/`event-test` onto `$(TEST)` with different flag sets, and `ci-pytest` runs `pytest -n auto -v $(srcdir)/http`.
- Memory-behaviour is a first-class test axis: debug builds honour `CURL_MEMDEBUG` (log file) and `CURL_MEMLIMIT` (fail the Nth allocation) set up in `memory_tracking_init()`, with tests/memanalyze.pl and tests/memanalyzer.pm analysing the logs and test cases asserting allocation ceilings.
- Debug builds expose test-only hooks the suite depends on: `CURL_FORBID_REUSE`, `global->test_duphandle` (perform on a `curl_easy_duphandle()` copy), `global->test_event_based` (`curl_easy_perform_ev()` / libuv-driven `parallel_event()`), and `CURL_UPLOAD_SIZE`.
- Source-level checks run as their own CI workflow: `scripts/checksrc-all.pl` enforces the style guide, and the same job runs spellcheck, `scripts/cmakeopts.sh`, REUSE licence checks and other linters with `permissions: {}` and pinned action SHAs.
- Documentation is tested too: `make check` includes `check-docs` (docs/libcurl) and `examples` (docs/examples, 129 compilable C programs), with dedicated checkdocs.yml and checkurls.yml workflows.
- The CI matrix is what covers the backend combinatorics: linux.yml pins and builds against AWS-LC, BoringSSL, LibreSSL, mbedTLS (current and previous), OpenSSL, Rustls, nghttp2, OpenLDAP and Fil-C, while http3-linux.yml, macos.yml, windows.yml, non-native.yml and linux-old.yml cover the rest.
- There is a coverage script (`scripts/coverage.sh`) and a CodeQL workflow, but coverage is not gated in the in-tree build targets — `make check` does not compute or threshold line coverage. (inferred)
- `configure-vs-cmake.yml` exists specifically to catch drift between the two build systems, which is the main risk of maintaining `configure.ac` (167KB) and `CMakeLists.txt` (93KB) in parallel. (inferred)