sindresorhus/got
github.com/sindresorhus/got ↗ · 129 files · 1.8 MB
Sindre's open source work is supported by the community.
Overview
Got 16 is a TypeScript HTTP client library for Node.js 22 and newer, distributed as native ESM. It serves applications that need buffered Promise requests, streaming uploads and downloads, pagination, configurable retries, and detailed transport control. Its distinctive architecture shares a Duplex request engine across both APIs while allowing custom instances, middleware handlers, lifecycle hooks, cache adapters, and request implementations.
System map
The runtime is a callable library instance built around a shared request engine. Factory code selects the outward Promise or stream interface, while Options centralizes configuration and Request owns transport lifecycle state. HTTP caching, DNS caching, and HTTP/2 pooling occupy separate state boundaries. Distribution and CI are defined by package.json, tsconfig.json, and one inspected GitHub Actions workflow.
- source/index.ts creates the default Got instance from new Options(), an empty handler list, and mutableDefaults:false, then exports the instance plus configuration, response, error, diagnostics, and factory APIs.
- create() constructs Request before dispatching handlers; its lastHandler calls flush() and returns either the Request stream or an asPromise() wrapper according to isStream.
- Options owns cloned internals, raw merge history, and explicit-header state; merge() runs init hooks before assigning validated option setters and rejects unknown option keys.
- Request extends Node Duplex and owns response metadata, redirect history, retry count, upload/download counters, active request, queued writes, timeout cancellation, and abort disposal.
- Default HTTPS transport uses https.request unless http2 is enabled; HTTP/2 auto() negotiates ALPN, chooses requestHttp2 for h2, and falls back to HTTP/1.1.
- HTTP cache wrappers are retained per storage adapter by cacheableStore; DNS state is owned by DnsCache, and HTTP/2 connections are pooled in Http2Agent.sessions with pending-session and queue state.
- The npm package is ESM, requires Node >=22, and publishes dist/source; build deletes dist and runs tsc, prepare invokes build, and tsconfig compiles source, test, and benchmark using an ES2024 target.
- npm test chains XO, tsc --noEmit, and AVA with the tsx ESM import hook; CI specifies npm install and npm run test on Ubuntu for Node 22, 24, and 26.
Feature catalog
The public surface includes ordinary HTTP calls, buffered body shortcuts, Duplex streams, and asynchronous pagination. Reliability controls include redirects, retry rules, timeouts, abort signals, and structured errors. Custom instances and hooks let applications adapt the client to particular services. Transport, caching, and diagnostic APIs expose capabilities needed by Node applications beyond a basic request wrapper.
- Got installs get, post, put, patch, head, delete, and query shortcuts for both Promise and stream calls; arbitrary configured methods remain available through Options.method.
- Buffered requests expose .json(), .text(), and .buffer(); responseType selects parsing, resolveBodyOnly selects the return shape, and buffer results are Uint8Array rather than necessarily Node Buffer.
- Uploads accept JSON, URL-encoded forms, native FormData, strings, byte views, Node streams, and iterable bodies; streams expose request, response, redirect, uploadProgress, and downloadProgress events.
- Reliability controls include two retries by default for configured methods/errors/statuses, exponential delay with noise and Retry-After handling, eight timeout settings, and AbortSignal cancellation.
- Redirects follow by default with maxRedirects:10; method rewriting distinguishes 303, cross-origin POST 301/302, optional methodRewriting, and body-preserving 307/308.
- got.extend() composes defaults and handlers; lifecycle hooks include init, beforeRequest, beforeRedirect, beforeRetry, beforeCache, afterResponse, and beforeError.
- got.paginate()/each yield items asynchronously and paginate.all collects them; callbacks control transformation, filtering, continuation, and next-request options, with count/request limits and backoff.
- Advanced integrations include TLS configuration, custom agents and connections, DNS caching, opt-in Unix sockets, cookie jars, HTTP cache adapters, diagnostic channels, and RequestError metadata.
Data flow
Inputs become normalized Options before native transport options are constructed. Request serializes payloads, prepares generated headers, and reconciles hook mutations before sending data. Incoming messages become decorated responses and either flow outward as stream chunks or accumulate for Promise parsing. Cache adapters, cookie jars, DNS stores, and HTTP/2 pools retain different kinds of state across calls.
- The public factory rejects url properties inside options objects; Options clones defaults, merges overrides, then assigns the URL so explicit searchParams replace URL query parameters.
- _finalizeBody() turns form into URLSearchParams text, JSON into stringifyJson output, and native FormData into a serialized body; generated content-type and known content-length headers accompany those transformations.
- _makeRequest() derives Basic authorization from username/password unless overridden or explicitly omitted, obtains cookie strings from the jar, then awaits beforeRequest hooks before reconciling changed headers and URL boundaries.
- createNativeRequestOptions() maps TLS settings, headers, agent, lookup, family, and createConnection into Node transport fields; Unix URLs become socketPath/path options only when enableUnixSockets permits them.
- _onResponseBase() attaches sanitized final URL, original request URL, redirects, request reference, cache origin, IP, and retry count; final responses are decompressed when enabled, while followed redirect bodies are drained without decompression.
- Promise mode gathers chunks with _setRawBody() into Uint8Array, optionally caches incremental UTF-8 decoding, then parseBody() produces text, JSON, or bytes; stream mode emits response and forwards readable data without that buffered completion gate.
- Cross-origin follow-up options are merged through mergeNextRequestOptions(), which handles explicit body replacement, cookie-jar removal, URL/query overrides, and stripping inherited credentials/body unless replacements were explicitly supplied.
- State retention is delegated: Set-Cookie values are written to the supplied jar in wire order, HTTP representations use the supplied cache adapter, and DnsCache stores resolved entries with clear-version checks that prevent stale asynchronous writes from surviving invalidation.
Behavior traces
Eight source walks cover instance initialization and the main user-visible execution paths. The Promise and stream paths share normalization and transport setup but differ in upload completion and response buffering. Redirect and retry paths preserve or replace request state under distinct rules. Cache reuse, cookie persistence, and pagination reconnect retained state to later requests.
- Initialization: source/index.ts creates Options → create(defaults) clones options/handlers → attaches extend, pagination, stream, and method shortcuts → freezes defaults unless mutableDefaults → exports got.
- JSON POST: got.post() → makeRequest() → new Request/new Options → handler chain/lastHandler → flush() → _finalizeBody() serializes JSON → _makeRequest(); response buffering precedes parseBody() and afterResponse hooks, then .json() returns the parsed body.
- Stream upload: got.stream.post() → makeRequest(...,true) → Request.flush() → _onRequest() → _sendBody(); without a configured body, a payload-capable stream waits for caller writes/end, then response data is exposed through the Duplex.
- Redirect: _onResponseBase() evaluates redirect policy → drains response → checks maxRedirects → clones Options/resolves Location → rewrites method/body and strips cross-origin state → runs beforeRedirect hooks → validates replayability → emits redirect and calls _makeRequest() again.
- Retry: _beforeError() normalizes failure and Retry-After → calculateRetryDelay() and optional calculateDelay → backoff timer → beforeRetry hooks → cleanup/body preservation → retry event; asPromise() rejects an unchanged consumed body or constructs and flushes the next Request.
- HTTP cache reuse: _makeRequest() selects eligible bodies → _prepareCache() creates/reuses a CacheableRequest wrapper → _createCacheableRequest() resolves a native request or cached ResponseLike → _onResponse() follows the normal response path and sets isFromCache; freshness/revalidation policy belongs to the dependency.
- Cookie persistence: _makeRequest() awaits getCookieString(URL) → sends generated Cookie unless omitted → _onResponseBase() writes each Set-Cookie sequentially → only then completes response/redirect handling; a later call consults the same caller-owned jar.
- Pagination: paginateEach() normalizes/validates callbacks → waits inter-page backoff → forces full-response resolution → calls got() → transform/filter/shouldContinue → yields items under limits → paginate() supplies follow-up options → applies cross-origin merge rules and refreshes pagination settings.
Testing signals
Tests and CI were inspected, not executed; no passing result or measured coverage is asserted. The inspected suite uses AVA, TypeScript assertions, local HTTP servers, and selected fake timers. Regression cases exercise transport identity, cross-origin behavior, buffering, caching, retries, and timeout budgets. The inventory is bounded, and explicit skipped or expected-failing cases limit what test presence establishes.
- AVA runs test/* excluding *.types.ts, treats TypeScript as modules, uses a ten-minute timeout, and disables worker threads; the script first runs XO and tsc --noEmit with tsx supplying ESM execution support.
- withServer creates a real HTTP server, optionally installs fake timers, builds a Got instance with that server's prefixUrl, and closes the server in finally; maintainer.md explicitly discourages nock because of process-wide interception.
- Promise tests assert request/response events, raw Uint8Array behavior with decompression disabled, and abort propagation through .json(); extend.types.ts checks inferred instance/default merges using expect-type under TypeScript checking.
- Cross-origin regressions distinguish POST 302 body removal from 307 body preservation with credential stripping; pagination tests check authorization removal and rejection of absolute Link URLs or in-place origin mutations.
- Retry tests assert a consumed generator produces only one upload and a replay error; request-timeout tests cover slow redirects, stalled downloads/uploads, and the intentional exclusion of initial beforeRequest hook time.
- Transport tests assert HTTP/2 pooled-socket reuse and separation by checkServerIdentity function identity; DnsCache tests inject resolvers to verify answer reuse and requested IPv4-family selection.
- Cache tests assert cacheable GET/POST reuse, QUERY bypass, 304 refresh of existing metadata, and uncached native FormData/Web-stream uploads; a separate test.failing documents missing new metadata from 304 responses.
- Inspection identified a skipped unhandled-body-error test and an expected-failing cache case; CI specifies Ubuntu Node 22/24/26 and npm test, while coverage is a separate c8 script with text/HTML/lcov output and is not invoked by the inspected workflow.
Risk map
These are source observations and explicitly labeled hypotheses, not confirmed vulnerabilities. Important boundaries include inherited credentials, redirect payload replay, diagnostic subscribers, local sockets, and caller-supplied transport settings. Async hooks and stream cleanup couple several request states, making seemingly small edits consequential. Buffering and a documented cache dependency limitation warrant targeted validation in a later execution-authorized task.
- Cross-origin 307/308 redirects intentionally preserve replayable bodies while removing inherited credentials; applications cannot equate credential stripping with preventing payload disclosure to a redirect destination.
- allowAbsoluteUrls:false restricts explicit inputs with prefixUrl, but internal redirects temporarily permit resolved absolute URLs; Unix sockets are separately opt-in and redirects cannot switch socket paths. These controls have distinct scopes.
- Diagnostic URLs have username/password removed, but request-start and response-start publications include full headers; subscribers therefore receive potentially sensitive Authorization, Cookie, or Set-Cookie values.
- Hypothesis: large or highly compressed responses can impose substantial memory pressure in Promise mode because _setRawBody() materializes chunks and concatenated bytes, while text/JSON paths can also retain decoded strings; strictContentLength validates transferred length rather than a decoded-size ceiling. (inferred)
- timeout.request starts only once initial setup is complete, so an initial beforeRequest hook can outlast that configured budget; a dedicated regression test explicitly expects a 1500ms hook to succeed with a 1000ms request timeout.
- Retry cleanup is tightly coupled to body identity: Request temporarily clears a replacement body during destroy(), preserves it for the next attempt, and asPromise rejects unchanged non-replayable bodies; incorrect ordering can destroy the replacement or reuse consumed input.
- HTTP/2 protocol/session identity includes certificate settings, rejectUnauthorized, checkServerIdentity, and lookup identity; removing these key components could defeat the separation that the checkServerIdentity regression test expects. (inferred)
- test/cache.ts explicitly attributes dropped newly introduced 304 metadata to http-cache-semantics and marks the scenario test.failing; this is a documented source-level limitation, not a failure reproduced during this inspection.
Extension points
Extension is primarily instance composition and callback-based customization. Handler middleware wraps the request interface, while lifecycle hooks operate at specific normalization and transport stages. Request functions, connection factories, agents, cookie jars, and cache adapters substitute infrastructure without replacing the public client. Pagination and JSON callbacks customize data interpretation and follow-up behavior.
- got.extend(...instancesOrOptions) merges option histories, concatenates handlers, carries mutableDefaults settings, and calls create() to produce a new instance.
- HandlerFunction receives Options and next(options); iterateHandlers dispatches them in order and preserves Promise helper descriptors on wrapped results. Stream handlers must return synchronously according to the instance documentation.
- Hook registration accepts only recognized hook events and function arrays; merging appends hooks, and init hooks can consume custom raw options into context before unknown-option validation.
- The request option can supply a custom request/response synchronously or asynchronously; undefined results fall back to the built-in transport. A beforeRequest hook can short-circuit later hooks by returning a request or response.
- Transport customization uses protocol-specific agents, createConnection, dnsLookup/dnsCache, and h2session; stable wrapped DNS lookup identity is deliberately maintained for HTTP/2 session reuse.
- Cache storage accepts Map-like adapters, with QuickLRU TTL translation; beforeCache hooks synchronously mutate cache metadata or return false to prevent storage, and Promise returns are rejected.
- Pagination customization replaces transform, filter, shouldContinue, and paginate callbacks; paginate can stop with false or supply updated options, and returned pagination settings affect subsequent iterations.
- parseJson/stringifyJson replace JSON codecs; cookieJar implements getCookieString/setCookie, with legacy callback jars promisified; diagnostics subscribers consume typed lifecycle messages through Node channels.
Contributor handoff
Begin at the factory, normalized options, shared Request engine, and Promise adapter before editing public behavior. Tests and CI were inspected, not executed, and this brief does not establish exhaustive coverage or passing checks. Focus changes on the owning layer while checking the corresponding Promise, stream, and follow-up paths. Preserve public typing, generated-header distinctions, and resource cleanup contracts.
- Read source/index.ts and create.ts for public dispatch, Options for normalization, Request.flush() for lifecycle entry, and asPromise() for buffered completion before changing request semantics.
- Public API changes must account for source/types.ts, RequestPromise types, exports, and compile-time assertions; RequestPromise is intentionally a type alias and does not support interface-merging augmentation.
- Preserve the public URL call contract and explicit-header semantics: url inside options objects is rejected, while sensitive follow-up headers explicitly supplied by callers are handled differently from inherited/generated headers.
- For later authorized validation, select relevant AVA files such as retry.ts, redirects.ts, pagination.ts, request-timeout.ts, cache.ts, or https.ts under the tsx import hook; npm run test is the repository's combined lint/type/runtime check and coverage has its own script.
- Use withServer/withServer.exec and real local test servers for regressions; maintainer.md identifies nock's process-wide interception as a test isolation trap.
- Preserve tab indentation, LF endings, ESM .js import suffixes, and npm's package-lock=false configuration; generated dist is ignored, and npm run build deletes it before compilation.
- Treat retry/body cleanup, cross-origin follow-up merges, and HTTP/2 pool identity as coupled edit boundaries; shared helpers are used by both afterResponse retry callbacks and pagination.
- Tests and CI in this brief were inspected, not executed; the Ubuntu-only workflow, skipped body-error test, and expected-failing 304 metadata test define concrete limits on evidence, so future agents must report actual runtime results separately.
Languages
| Language | Files | Share of source |
|---|---|---|
| TypeScript | 84 | 93% |
| Markdown | 27 | docs/config |
| JavaScript | 6 | 7% |
| JSON | 2 | docs/config |
| YAML | 2 | docs/config |
Start reading here
Probable source entrypoint: source/index.ts — inferred from the directory tree and manifest, not the upstream docs.
Source roots
source/ (27 files)
Source layout — source/
| Path | Files |
|---|---|
source/as-promise/ | 2 |
source/core/ | 22 |
source/create.ts | 1 |
source/index.ts | 1 |
source/types.ts | 1 |
Repository structure
| Top-level path | Files |
|---|---|
test/ | 57 |
source/ | 27 |
documentation/ | 26 |
.github/ | 7 |
benchmark/ | 2 |
media/ | 1 |
Docs found: documentation/1-promise.mddocumentation/10-instances.mddocumentation/2-options.mddocumentation/3-streams.mddocumentation/4-pagination.mddocumentation/5-https.mddocumentation/6-timeout.mddocumentation/7-retry.mddocumentation/8-errors.mddocumentation/9-hooks.md
Package
package.json — got v16.0.0, 50 declared dependencies.
@hapi/bourne@sindresorhus/is@sindresorhus/tsconfig@sinonjs/fake-timers@types/benchmark@types/express@types/node@types/pem@types/readable-stream@types/request@types/sinon@types/sinonjs__fake-timers
CI & config entrypoints
.github/workflows/main.yml— GitHub Actions workflowpackage.json— npm scripts: build, prepare, release, test, test:coverage
Structural health signals
Average score 8.2/10 across 90 analyzed source files; 8 hotspots surfaced.
source/core/index.ts— 2518 lines, score 5.2 (Large file, Long function/block)source/core/utils/http2-client.ts— 1718 lines, score 5.2 (Large file, Long function/block)test/redirects.ts— 3167 lines, score 5.5 (Large file, Long function/block)
These are deterministic review leads — size, churn, and structural signals — not proof of a bug in this project.
Recent history
e1d87d2ced012026-09-18 — More testsdf33cdb510c22026-09-18 — Replace the request timeout budget arithmetic with one per-request timerc36ad51551332026-09-18 — Strip one leading byte order mark when decoding UTF-8 and UTF-16LE text0deff95160aa2026-09-18 — Add regression tests for redirect bodies, retry delays, cache bypass, and option resetse1ff206d2a9e2026-09-18 — Share the next-request option merging between afterResponse retries and pagination