CodeVetter · Repo Unpack · Data flow
httpie/cli
Input arrives from four boundaries -- argv, stdin, files referenced by request items, and the JSON config/session files -- and is normalized inside `HTTPieArgumentParser.parse_args`, which mutates a single `argparse.Namespace` through an ordered chain of `_process_*` methods. `RequestItems.from_args` fans the raw `KeyValueArg` list into four typed multi-dicts (headers, data, params, files) plus a multipart ordering dict. `client.collect_messages` then translates the namespace into `requests.Request` kwargs, applies session headers/cookies/auth, prepares the request, optionally deflates it, and yields message objects lazily. Output flows back through `OutputOptions.from_message` into `write_message`, which picks a stream class (Raw/Encoded/Pretty/BufferedPretty) and writes bytes to `env.stdout.buffer`. On the way out, session cookies and headers are re-materialized and written back to JSON.
- The parse pipeline is strictly ordered and order-dependent: `_apply_no_options` -> `_process_request_type` -> `_process_download_options` -> `_setup_standard_streams` -> `_process_output_options` -> `_process_pretty_options` -> `_process_format_options` -> `_guess_method` -> `_parse_items` -> `_process_url` -> `_process_auth` -> `_process_ssl_cert`, followed by stdin/`--raw` body ingestion.
- `config.default_options` is prepended to argv (not merged post-parse), so user defaults behave exactly like typed flags and can be individually reverted with the generic `--no-OPTION` mechanism implemented in `_apply_no_options`.
- Method inference is data-driven: with no METHOD, POST is chosen if there is stdin/`--raw` input else GET; when the first positional is not alphabetic, the parser re-interprets it as a request item and shifts URL/method, then chooses POST if any item separator is in `SEPARATOR_GROUP_DATA_ITEMS`.
- In JSON mode, nested-JSON items are split out first via `split_iterable` and interpreted as a batch (so `[]` append semantics work across items), while all remaining items are processed one-by-one into their target dicts.
- Headers use `HTTPHeadersDict`, a case-insensitive multidict where `add(key, None)` means 'explicitly unset'; `finalize_headers` later converts `None` values for urllib3's SKIPPABLE_HEADERS into `SKIP_HEADER` and strips/encodes all other values.
- Because `requests`' PreparedRequest collapses duplicate headers, `apply_missing_repeated_headers` re-injects the original repeated values after preparation, skipping any field that was overridden along the way.
- Default headers are computed from the request shape: User-Agent is always `HTTPie/<version>`, JSON mode adds `Accept: application/json, */*;q=0.5` and `Content-Type: application/json`, and form mode without files adds `application/x-www-form-urlencoded; charset=utf-8`.
- Session headers are merged beneath CLI headers (`headers.update(base_headers)` then `headers.update(args.headers)`), so a header typed on the command line always wins over the stored session value.
- Request bodies are normalized by `prepare_request_body`: str/bytes become bytes, `RequestDataDict` becomes urlencoded bytes, file-likes go through `_prepare_file_for_upload`, and `--chunked` wraps everything in a `ChunkedUploadStream`/`ChunkedMultipartUploadStream` generator.
- Upload progress is echoed back to stdout through a callback loop: `program()` passes `request_body_read_callback` into `collect_messages`, which threads it into `prepare_request_body`, which wraps `file.read` so each chunk is re-emitted via `write_raw_data`.
- Redirects are handled manually rather than by requests: `allow_redirects` is hard-set to False in `make_send_kwargs`, and `collect_messages` walks `response.next` itself so that each hop can be yielded for `--all`/`--history-print` and counted against `--max-redirects`.
- Response bytes are decoded adaptively: `EncodedStream` uses the declared charset if present, otherwise accumulates `charset_normalizer` guesses and locks one in after `ENCODING_GUESS_THRESHOLD` (3) consecutive agreeing guesses; bodies containing NUL bytes raise `BinarySuppressedError` and print the 'binary data not shown' notice instead.
- Session write-back materializes runtime state into serializable JSON: `post_process_data` converts the `RequestsCookieJar` into a list of dicts keeping only name/expires/path/value/domain/secure, and the header multidict into a list of `{name, value}` objects, each passed through the corresponding legacy exporter.
- Expired cookies are pruned on the way out: every response's `Set-Cookie` is scanned by `get_expired_cookies`, which translates `max-age` into `expires` (a workaround for a requests bug) and returns name/path pairs that `Session.remove_cookies` deletes before saving.
- Headers written to a session deliberately drop request-specific fields: anything prefixed `Content-` or `If-`, the HTTPie-generated User-Agent, and `Cookie` (whose morsels are diverted into the cookie jar instead).