# Extension points — firecrawl/firecrawl

> Canonical page: https://codevetter.com/unpack/firecrawl/extension-points

# Extension points — firecrawl/firecrawl

Scanned commit 556c12c9d293bc7392feb008946944d54068d04c (2026-09-21). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

The cleanest extension seams are the scrape engine registry (add an Engine string, handler, MRT and feature/quality entry), the transformer/postprocessor lists, and the route tables (v2Router wiring controllers through shared middlewares). NuQ job modes ('single_urls', 'kickoff', 'kickoff_sitemap') dispatch in processJobWithTracing, search providers chain in search/index.ts, threat-protection providers plug behind lib/threat-protection, and exchange eligibility is isolated in lib/exchange.ts. New env config goes in the Zod schema; new team flags ride TeamFlags in the ACUC.
- Adding a scrape engine = extend the Engine union, append to engines[] behind an env gate, and fill engineHandlers, engineMRTs and engineOptions (feature support + quality) — buildFallbackList and the waterfall pick it up automatically. — [apps/api/src/scraper/scrapeURL/engines/index.ts#L51-103](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/scrapeURL/engines/index.ts#L51-L103) [apps/api/src/scraper/scrapeURL/engines/index.ts#L193-239](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/scrapeURL/engines/index.ts#L193-L239) [apps/api/src/scraper/scrapeURL/engines/index.ts#L241-608](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/scrapeURL/engines/index.ts#L241-L608)
- Feature flags are declarative: the featureFlags tuple plus featureFlagOptions priorities drive which engines qualify; adding a flag requires updating the union, every engine's features map, and the format/options schema in controllers/v2/types.ts. — [apps/api/src/scraper/scrapeURL/engines/index.ts#L105-149](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/scrapeURL/engines/index.ts#L105-L149)
- New output formats plug in as transformers: executeTransformers runs modules like llmExtract, deterministicJson, diff (changeTracking), menu, product, audio, video, sendToSearchIndex; postprocessors (youtube) run after. — [apps/api/src/scraper/scrapeURL/transformers/index.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/scrapeURL/transformers/index.ts) [apps/api/src/scraper/scrapeURL/postprocessors/index.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/scrapeURL/postprocessors/index.ts) [apps/api/src/scraper/scrapeURL/index.ts#L65-66](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/scrapeURL/index.ts#L65-L66)
- Routes are registered flat in routes/v2.ts with per-route middleware composition; a new endpoint needs a controller under controllers/v2, a RateLimiterMode in types.ts, and the standard auth/credits/blocklist middleware stack. — [apps/api/src/routes/v2.ts#L189-242](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/routes/v2.ts#L189-L242) [apps/api/src/routes/shared.ts#L271-326](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/routes/shared.ts#L271-L326)
- NuQ job dispatch is a mode switch: ScrapeJobData.mode ('single_urls'|'kickoff'|'kickoff_sitemap') selects processJob/processKickoffJob/processKickoffSitemapJob in processJobWithTracing — new job kinds extend the union in types.ts and add a branch. — [apps/api/src/types.ts#L33-89](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/types.ts#L33-L89) [apps/api/src/services/worker/scrape-worker.ts#L1818-1840](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/services/worker/scrape-worker.ts#L1818-L1840)
- Search providers chain by config: search/index.ts tries fire_engine_search when FIRE_ENGINE_BETA_URL is set, then searxng_search when SEARXNG_ENDPOINT is set, then ddgSearch — a new provider slots into that fallback chain. — [apps/api/src/search/index.ts#L29-60](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/search/index.ts#L29-L60) [apps/api/src/search/searxng.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/search/searxng.ts) [apps/api/src/search/v2/ddgsearch.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/search/v2/ddgsearch.ts)
- URL-specific behavior is table-driven: urlSpecificParams overrides options per domain, engine-forcing (getEngineForUrl/initializeEngineForcing) pins engines per host, and the blocklist (initializeBlocklist/isUrlBlocked) is a warmed data file — all three are data-shaped extension points. — [apps/api/src/scraper/scrapeURL/lib/urlSpecificParams.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/scrapeURL/lib/urlSpecificParams.ts) [apps/api/src/scraper/WebScraper/utils/engine-forcing.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/WebScraper/utils/engine-forcing.ts) [apps/api/src/scraper/WebScraper/utils/blocklist.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/scraper/WebScraper/utils/blocklist.ts)
- Threat protection is provider-pluggable: lib/threat-protection resolves a policy per team/request and checkUrl consults providers (Google Web Risk normal/ZDR-synced modes, Zscaler ZIA), with org config exposed via /v2/team/threat-protection routes. — [apps/api/src/lib/threat-protection/index.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/lib/threat-protection/index.ts) [apps/api/src/lib/threat-protection/request.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/lib/threat-protection/request.ts) [apps/api/src/config.ts#L123-150](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/config.ts#L123-L150)
- Billing features are keyed by Autumn feature IDs (CREDITS_FEATURE_ID, SEARCH_CREDITS_FEATURE_ID, featureIdForBillingEndpoint) — new metered endpoints pass their featureId through checkCreditsMiddleware's second arg. — [apps/api/src/routes/shared.ts#L38-41](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/routes/shared.ts#L38-L41) [apps/api/src/routes/v2.ts#L194](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/routes/v2.ts#L194) [apps/api/src/services/autumn/autumn.service.ts](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/services/autumn/autumn.service.ts)
- SDKs are independently versioned packages under apps/*-sdk with per-language publish workflows; the API surface they consume is documented by openapi.json/v1-openapi.json/openapi-v0.json in apps/api. — [apps/api/package.json](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/package.json) [.github/workflows/publish-js-sdk.yml](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/.github/workflows/publish-js-sdk.yml) [apps/api/openapi.json](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/openapi.json)
- Config extension is centralized: new env vars require a Zod entry in configSchema (with helpers like emptyStringAsUndefined/delimitedList) and are then read as config.X everywhere; MCP action-log config even validates at startup via getMcpActionLogConfigErrors. — [apps/api/src/config.ts#L1-70](https://github.com/firecrawl/firecrawl/blob/556c12c9d293bc7392feb008946944d54068d04c/apps/api/src/config.ts#L1-L70)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)