CodeVetter · Repo Unpack · Data flow
curl/curl
Input boundaries are argv, `.curlrc`/`-K` config files, environment variables, `.netrc`, stdin uploads, and the network. argv is normalised into a linked list of `struct OperationConfig` (one per `--next`), which `config2setopts.c` translates wholesale into `curl_easy_setopt()` calls; from there the state owner is `struct Curl_easy` (`data->set` for user intent, `data->state`/`data->req` for per-request state). The URL becomes a `CURLU` via lib/urlapi.c, resolves to a `struct Curl_scheme`, and is matched against the connection pool before a filter chain is built. Bytes in from the network flow up through `Curl_xfer_write_resp()` → `Curl_client_write()` → an ordered writer stack → `CURLOPT_WRITEFUNCTION`; bytes out flow down through the mirror-image client reader chain. Output boundaries are the write callbacks (which the tool points at files or stdout), the header callback, the progress callback, and the file-backed caches flushed at cleanup.
- argv is consumed by `parse_args()`, which loops over argv, calls `getparameter()` per flag, treats bare arguments as `--url`, and on `PARAM_NEXT_OPERATION` allocates a fresh `OperationConfig` and links it onto `global->last` — so `--next` literally forks the config list.
- Config files are a second input boundary parsed by the same code path: `operate()` calls `parseconfig(NULL, CONFIG_MAX_LEVELS, &curlrc_path)` for `.curlrc` before argv, and `-K` re-enters it, with recursion bounded by `CONFIG_MAX_LEVELS`.
- The config→library translation is centralised in src/config2setopts.c, split into `ssh_setopts()`, `ssl_ca_setopts()`, `ssl_setopts()`, `cookie_setopts()`, `httpsig_setopts()`, `http_setopts()` and friends, all using `my_setopt_*` wrappers so the same call sites feed both libcurl and the `--libcurl` code generator.
- Library-side, options land in `struct Curl_easy` guarded by `magic == CURLEASY_MAGIC_NUMBER`; `lib/setopt.c` fans out into typed setters (`setopt_long_bool`, `setopt_long_net`, `setopt_long_ssl`, `setopt_long_proxy`, `setopt_long_http`) rather than one giant switch.
- URLs are parsed by `parseurl()` in lib/urlapi.c into a `CURLU`, with behaviour gated by flags such as `CURLU_GUESS_SCHEME`, `CURLU_DEFAULT_SCHEME`, `CURLU_NO_AUTHORITY`, `CURLU_DISALLOW_USER`, `CURLU_URLENCODE` and `CURLU_PATH_AS_IS`; the same parser backs the public `curl_url_*` API.
- The parsed scheme is only accepted for a connection if it passes `url_set_conn_scheme()`, which requires a non-NULL `run` vtable and membership in `data->set.allowed_protocols` (and `redir_protocols` when following a redirect) — this is the choke point that makes `--proto`/`--proto-redir` effective.
- Before dialling out, libcurl asks the connection pool for a reusable `connectdata` through `Curl_cpool_find()` with a match callback; reuse eligibility consults PROTOPT_SSL/PROTOPT_SSL_REUSE/PROTOPT_CONN_REUSE and compares origins and protocol families.
- Received bytes enter the client-writer chain via `Curl_client_write()`, which lazily builds the stack (`Curl_cwt_out` at CURL_CW_CLIENT, `cw_download` at CURL_CW_PROTOCOL, `cw_raw` at CURL_CW_RAW) and inserts each new writer first within its phase.
- Writer ordering is a fixed phase enum — RAW → TRANSFER_DECODE → PROTOCOL → BEFORE_DECODE → CONTENT_DECODE → CLIENT — so chunked decoding, size accounting, and gzip/brotli/zstd decompression always run in that order before the application callback.
- `cw_download_write()` is where `--max-filesize` is enforced and where excess body bytes beyond `req.maxdownload` cause `connclose()`; exceeding the limit returns `CURLE_FILESIZE_EXCEEDED` from inside the writer chain.
- Upload data flows through the mirror-image client-reader chain with phases CLIENT → CONTENT_ENCODE → PROTOCOL → TRANSFER_ENCODE → NET, entered via `Curl_client_read()`; HTTP's Expect: 100-continue is itself implemented as a reader (`cr_exp100_read`).
- On the tool side the terminal output boundary is `tool_cb_wrt.c`'s write callback writing into a `struct OutStruct` (stream, filename, `fopened`, `init` offset), with `setup_outfile()` deciding stdout vs. file vs. `--out-null` and opening in append mode when resuming.
- Response headers reach the tool through `tool_cb_hdr.c`, where `parse_filename()` derives a local name from `Content-Disposition:` (or `Location:`) by trimming quotes, query/fragment, CR/LF, and any path prefix before `/` or `\`.
- Cross-transfer state is shared deliberately: `share_setup()` enables `CURL_LOCK_DATA_COOKIE`, `DNS`, `SSL_SESSION`, `PSL`, `HSTS` on the `CURLSH`, and adds `CURL_LOCK_DATA_CONNECT` only in serial mode because parallel mode uses the multi handle's own pool.
- Persistence is flushed at teardown rather than continuously: `Curl_flush_cookies()` writes the jar through `cookie_output()`, and `Curl_fopen()` performs the durable write as O_EXCL temp file + `fchmod` clone of the original mode + rename.