# Contributor handoff — firecrawl/firecrawl

> Canonical page: https://codevetter.com/unpack/firecrawl/agent-handoff

# Contributor handoff — firecrawl/firecrawl

Scanned commit 556c12c9d293bc7392feb008946944d54068d04c (2026-09-21). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Work happens almost entirely in apps/api. Read AGENTS.md/CLAUDE.md first — they mandate E2E 'snips' over unit tests, `pnpm harness jest` (never pnpm start), and fixing knip failures rather than bypassing hooks. Safe edit boundaries: a new engine touches only engines/* plus the registry; a new endpoint touches controllers/v2 + routes/v2.ts + a RateLimiterMode; a new transformer touches transformers/. Danger zones: the auth/credit middleware order, ZDR propagation, and anything changing job payload shape (NuQ rows in flight). Always add env config to the Zod schema and gate tests by capability flags.
- AGENTS.md is the contract: write snips (1 happy + 1+ failure paths), use scrapeTimeout from ./lib for scrape timeouts, gate fire-engine tests on !TEST_SUITE_SELF_HOSTED and AI tests on OPENAI_API_KEY/OLLAMA_BASE_URL, run via `pnpm harness jest`, push a PR and let CI run the full suite. — [AGENTS.md](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/AGENTS.md)
- CLAUDE.md adds: never bypass the pre-commit knip check (fix unused exports/files even if preexisting); package.json wires husky+lint-staged+knip so commits enforce prettier formatting and dead-code hygiene. — [CLAUDE.md](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/CLAUDE.md) [apps/api/package.json#L43-44](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/package.json#L43-L44) [apps/api/knip.config.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/knip.config.ts)
- Read first for scrape work: controllers/v2/types.ts (2825-line zod schema for all request/response types), scraper/scrapeURL/index.ts (1926-line pipeline), engines/index.ts (registry), and lib/error.ts + scrapeURL/error.ts for the TransportableError taxonomy. — [apps/api/src/controllers/v2/types.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/controllers/v2/types.ts) [apps/api/src/scraper/scrapeURL/index.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/scrapeURL/index.ts) [apps/api/src/lib/error.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/lib/error.ts)
- Middleware order is load-bearing: requestTiming → authMiddleware → countryCheck → checkCredits → blocklist → controller; reordering or omitting one (e.g. on a new route) silently drops auth, geo-restriction, billing or blocklist enforcement. — [apps/api/src/routes/v2.ts#L235-290](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/routes/v2.ts#L235-L290) [apps/api/src/routes/shared.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/routes/shared.ts)
- Zero-data-retention must be threaded: ZDR suppresses OTel recording, GCS/index writes, and URL logging; any new code path that touches a scrape must check internalOptions.zeroDataRetention / getScrapeZDR before persisting or tracing. — [apps/api/src/controllers/v2/scrape.ts#L63-72](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/controllers/v2/scrape.ts#L63-L72) [apps/api/src/lib/zdr-helpers.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/lib/zdr-helpers.ts) [apps/api/src/services/worker/scrape-worker.ts#L1776-1783](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/services/worker/scrape-worker.ts#L1776-L1783)
- Tests require a running stack: `pnpm harness` spawns api+workers+containers; snips hit TEST_API_URL with TEST_API_KEY='test'/TEST_TEAM_ID='bypass' in self-host mode — do not hand-start the server, and scope test runs to the relevant suite since the full suite is slow. — [AGENTS.md](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/AGENTS.md) [apps/api/src/__tests__/snips/lib.ts#L9-30](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/__tests__/snips/lib.ts#L9-L30) [apps/api/package.json#L17-22](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/package.json#L17-L22)
- Naming traps: _addScrapeJobToBullMQ actually enqueues onto NuQ (BullMQ naming is vestigial); 'snips' means E2E tests; 'ACUC' is the auth/credit chunk; 'NuQ' queues live in Postgres tables prefixed nuq.; job 'mode' (not endpoint) selects worker behavior. — [apps/api/src/services/queue-jobs.ts#L154-226](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/services/queue-jobs.ts#L154-L226) [apps/api/src/types.ts#L33-89](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/types.ts#L33-L89) [apps/api/src/services/worker/nuq.ts#L1755-1760](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/services/worker/nuq.ts#L1755-L1760)
- Error contract: controllers return {success:false, error} with status codes mapped per TransportableError code in scrape.ts's catch block — new error codes need an explicit branch (e.g. SCRAPE_DNS_RESOLUTION_ERROR→200, SCRAPE_NO_CACHED_DATA→404) or they surface as UNKNOWN_ERROR 500s. — [apps/api/src/controllers/v2/scrape.ts#L462-585](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/controllers/v2/scrape.ts#L462-L585) [apps/api/src/lib/error.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/lib/error.ts)
- Env/config changes: add the var to the Zod schema in config.ts (empty-string handling matters — use emptyStringAsUndefined), reference it via config.X, and update docker-compose.yaml common-env + CI workflow envs if the feature must work self-hosted. — [apps/api/src/config.ts#L13-17](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/config.ts#L13-L17) [docker-compose.yaml#L24-60](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/docker-compose.yaml#L24-L60) [.github/workflows/test-server.yml#L36-62](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/.github/workflows/test-server.yml#L36-L62)
- In-flight job compatibility: NuQ rows are JSONB payloads consumed by separate worker binaries, so changing ScrapeJobData/internalOptions shape requires backward-compat handling (the code already guards with ?? fallbacks like job.data.internalOptions?.orgId ?? null). — [apps/api/src/main/runWebScraper.ts#L28-34](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/main/runWebScraper.ts#L28-L34) [apps/nuq-postgres/nuq.sql#L44-60](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/nuq-postgres/nuq.sql#L44-L60) [apps/api/src/types.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/types.ts)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)