effect-ts/effect
github.com/effect-ts/effect ↗ · 2581 files · 31.9 MB
Effect Effect is a library for building robust, maintainable, type-safe, and production grade applications in TypeScript.
Overview
Effect is a TypeScript library ecosystem for developers building applications with typed failures, dependency injection, structured concurrency, schema validation, and managed resources. At commit 6389d9ac64c0f62ccc8b575fb9afc65fc104e814, the repository contains Effect 4.0.0 plus synchronized platform, SQL, AI-provider, UI-state, telemetry, and testing integrations. Its distinctive architecture carries success, error, and service requirements through composable APIs, then connects them to platform implementations through Context services and Layers.
System map
The repository is a pnpm workspace organized around the core effect package and technology-specific integrations. Public entrypoints expose TypeScript source during workspace development and compiled JavaScript when published. Context, Layer, fibers, and scopes provide the shared execution architecture across these packages. Build, verification, generation, and release workflows are repository-level concerns rather than an application deployment stack.
- The core effect manifest declares version 4.0.0, exports src/index.ts and subsystem entrypoints such as effect/http and effect/sql, and redirects published exports to dist while blocking internal paths.
- Context owns typed service maps; Layer<ROut, E, RIn> constructs service contexts with scoped acquisition; ManagedRuntime caches that context and owns separate layer and execution-fiber scopes.
- Platform boundaries are replaceable services: NodeHttpServer adapts Node request, response, and upgrade events, while FetchHttpClient supports Web Fetch environments and an injectable Fetch reference.
- Storage is supplied through implementations rather than one repository-wide database: KeyValueStore provides memory, filesystem, and SQL layers; the Node SQLite layer supplies both SqliteClient and generic SqlClient.
- Separate integrations provide Anthropic, OpenAI, OpenAI-compatible, OpenRouter, and TypeSafe AI backends; React, Solid, and Vue atom bindings; and OpenTelemetry support.
- Root build runs TypeScript project references, recursive package builds, and AI-document copying; package builds compile then Babel-annotate pure calls. Root scripts also expose lint, circular-dependency checks, API diff, and runtime/type performance harnesses.
- Check CI combines static, type, build, bundle, runtime, and documentation jobs; bundle-comment consumes its validated stats artifact, while nightly ai-codegen generates provider updates and conditionally opens a PR.
- Release packaging uses Changesets with a fixed synchronized package group; main releases can publish and dispatch website deployment, snapshots publish PR packages after fork gating, and release-queue manages eligible next-release branches.
Feature catalog
The user-facing product is a developer API ecosystem spanning execution, data modeling, network services, persistence, and integrations. CLI commands and application endpoints are defined by consumers through typed builders. Distributed entities, workflows, AI tools, and reactive UI state extend the same service model. These eight groups describe the inspected capabilities; they are a bounded catalog rather than an enumeration of every exported API.
- Effect describes typed, lazy programs with retry, interruption, concurrency, and resource management; Stream adds incremental sources, transformation, buffering, timing, and consumption.
- Schema supports validation, decoding, encoding, refinements, transformations, defaults, classes, JSON Schema generation, test-data generation, equivalence, optics, and derived differs.
- HTTP APIs combine schema-backed handlers, middleware, and optional OpenAPI GET output; FetchHttpClient sends ordinary or streaming request bodies, and HttpStaticServer supports files, directory indexes, and conditional SPA fallback.
- RpcGroup composes tagged RPC definitions and handlers; Cluster Entity adds stable entity IDs, shard selection, sharded clients, and handler or mailbox registration.
- CLI consumers receive argument and flag parsing, command trees, scoped global flags, help/error rendering, and an interactive wizard that can optionally execute the constructed command.
- Persistence includes schema-backed key-value storage and multiple SQL drivers, including Cloudflare D1 and Durable Objects SQLite; workflows expose execute, poll, interrupt, resume, and registration with explicit idempotency keys.
- LanguageModel exposes text, structured-object, and streaming generation; Toolkit binds tool schemas to validated handlers; McpServer exposes tools, resources, templates, prompts, completions, and notifications over stdio or HTTP.
- React atom hooks support selected reads, writes, refresh, and Promise/Exit write modes; corresponding Solid and Vue packages exist, and NodeSdk conditionally enables tracing, metrics, and logging when processors or readers are supplied.
Data flow
Inputs enter through execution bridges, HTTP requests, provider requests, storage calls, and framework hooks. Context supplies services, schemas transform external representations, and scoped implementations perform platform operations. Runtime contexts, router tables, databases, and atom registries own different kinds of state. Output boundaries include Promise results, encoded HTTP responses, SQL rows, persisted bytes, and UI subscriptions.
- Effect creation describes work; execution crosses runForkWith into FiberImpl.evaluate. runPromiseExitWith observes the fiber's Exit, and runPromiseWith returns its value or throws causeSquash for failure.
- ManagedRuntime's first context request builds the Layer once through a shared buildFiber, caches the resulting Context, and provides it to programs; subsequent Promise runs use runPromiseWith(cachedContext).
- HttpRouter matches method and URL, retries HEAD as GET if necessary, then places parsed search parameters and route parameters in the fiber Context before evaluating the route handler.
- handlerToHttpEffect decodes configured params, headers, query, and payload slots before calling the handler, then encodes success bodies and optional response headers. Raw handlers skip payload parsing, and returned HttpServerResponse values bypass result encoding.
- OpenAI generation converts provider options through prepareMessages, prepareTools, and prepareResponseFormat, sends POST /responses through a transformed HttpClient, schema-decodes the response, and converts it into Effect AI response parts.
- toSchemaStore builds a JSON codec: set encodes an application value before backend storage, while get reads a string and decodes it into Option.some; absent backend values become Option.none.
- SQL key-value writes encode strings to bytes and persist a value_type discriminator. Statement compiles SQL and parameters, acquires or borrows a connection, and SQLite binds parameters through StatementSync.all/run; reads reconstruct strings or Base64 binary representations.
- RegistryProvider creates one AtomRegistry for a React subtree. useAtomValue reads through that registry, optionally mapping an atom, while Promise-mode setters write then wait through AtomRegistry.getResult.
Behavior traces
Three source-level flows were followed to their completion boundaries: managed-runtime startup/execution/disposal, HTTP API registration/request handling, and schema-backed SQLite persistence. Each trace includes the functions responsible for state ownership and relevant conditional branches. These are execution-order reconstructions from source, not observed runs. Documentation's Hono integration provides a concrete consumer of the managed-runtime bridge.
- Runtime startup: ManagedRuntime.make creates a MemoMap and sequential root/layer scopes plus a parallel fiber scope; the first contextEffect starts Effect.runFork → Layer.buildWithMemoMap → layer.build and caches the resulting Context.
- Runtime primary action: runtime.runPromise chooses provide(contextEffect, program) before initialization or runPromiseWith(cachedContext) afterward; runPromiseExitWith → runForkWith → FiberImpl.evaluate leads to a Promise resolved from Exit or rejected from causeSquash.
- Runtime completion: dispose → Effect.runPromise(disposeEffect) invalidates contextEffect, clears cachedContext, and calls Scope.close. The Hono example invokes disposal on SIGINT/SIGTERM; regression tests assert request cleanup finishes before layer resources close.
- HTTP registration: HttpApiBuilder.group calls build(makeHandlers(group)), converts handlers via handlerToRoute, and stores routes under group.key; HttpApiBuilder.layer collects those group services and calls router.addAll, dying with a diagnostic if a group is missing.
- HTTP request: HttpRouter.asHttpEffect → router.find → request/route Context installation → middleware-wrapped handlerToHttpEffect → slot decoders → handler → success encoder. Missing routes fail as RouteNotFound; raw responses return directly, and declared handler errors pass through encodeError.
- SQLite storage initialization: SqliteClient.layer → make → makeConnection opens DatabaseSync, registers db.close as a scope finalizer, configures busy_timeout and optional WAL, and creates the generic client; KeyValueStore.layerSql obtains it and creates the dialect-specific table.
- Persistence write: SchemaStore.set → schema encode → KeyValueStore.set → UTF-8 bytes/value_type → SQLite upsert.unprepared → Statement.withConnection → compile → executeUnprepared → prepare → runStatement → StatementSync.run/all.packages/effect/src/persistence/KeyValueStore.ts#L799-L799packages/effect/src/persistence/KeyValueStore.ts#L580-L585packages/effect/src/persistence/KeyValueStore.ts#L661-L675packages/sql/sqlite-node/src/SqliteClient.ts#L269-L271packages/effect/src/sql/Statement.ts#L1363-L1375packages/effect/src/sql/Statement.ts#L1432-L1437packages/sql/sqlite-node/src/SqliteClient.ts#L154-L158packages/sql/sqlite-node/src/SqliteClient.ts#L166-L184
- Persistence read: SchemaStore.get → SQL SELECT by key → first-row value_type decoding → schema JSON decode → Option.some; missing rows return undefined at the backend and Option.none at the schema boundary, while invalid stored type discriminators fail with KeyValueStoreError.
Testing signals
Tests and CI were inspected, not executed; no passing result or coverage measurement is asserted. The repository separates Vitest runtime tests, Tstyche compiler contracts, and generated documentation tests. Representative tests exercise lifecycle ordering, HTTP contracts, provider request mapping, and SQLite storage/transactions. Runtime conditions and explicit integration flags limit which tests a particular invocation includes, and the inspected inventory is not exhaustive coverage.
- Vitest discovers package test/**/*.test.{ts,tsx}, uses shared setup and concurrent sequencing, and offers V8 HTML coverage. @effect/vitest re-exports Vitest and adds effectful/property-test contracts with interruption-aware timeouts.
- ManagedRuntime.test.ts asserts build memoization, Context provision, shared MemoMap reuse, await-using cleanup, interruption on disposal, request-before-resource cleanup ordering, and finalizer completion under interrupted disposal.
- The Node HTTP API tests construct real test-server layers and generated clients, checking catch-all routing and middleware error encoding on both server and client boundaries.
- SQLite KeyValueStore tests reuse a serialized suite for default and custom tables, covering overwrite/size, binary representation, UTF-8 strings, removal, and clear. Client tests separately cover rollback, deferred-commit recovery, busy-timeout configuration, contention, and readonly transactions.
- Inspected OpenAiClient tests assert Bearer headers, default/custom URLs, organization/project headers, and transform ordering using captured requests and synthetic responses; those assertions do not establish live-provider compatibility.
- Tstyche discovers .tst files in package typetest trees and includes Effect.isEffect narrowing assertions. Documentation tests use @effect/doctest/Plugin with includeSource covering top-level and nested package source.
- Inspected Check CI enables EFFECT_INTEGRATION_TESTS=1 and defines Node, Deno, and Bun test commands; separate jobs run lint/circular/document-generation checks, TypeScript/Deno/Tstyche checks, distribution builds, and documentation tests. These definitions were not executed in this investigation.
- Ordinary runs exclude integration tests unless EFFECT_INTEGRATION_TESTS=1; cluster integration additionally requires Node and EFFECT_CLUSTER_TESTS=1 and has a manual workflow. Bun excludes named core HTTP/schema/reactivity tests, and platform projects depend on the detected runtime.
Risk map
The inspected security-sensitive boundaries include credential handling, process execution, filesystem operations, native SQLite, dynamic compilation, and privileged CI. Several APIs intentionally expose caller-selected capabilities and therefore require appropriate consumer configuration. Two findings are explicitly labeled hypotheses requiring reproduction: concurrent schema-store modification and static-file symlink containment. None of these observations establishes a confirmed vulnerability or a repository-wide absence of safeguards.
- securityDecode extracts HTTP/API-key/Basic credentials and returns empty credentials for missing or malformed inputs; supplied security middleware determines acceptance. OpenAiClient separately reveals Redacted credentials to construct outbound headers and accepts a caller-selected apiUrl.
- NodeChildProcessSpawner forwards command and argument arrays to node:child_process.spawn after resolving cwd, environment, and stdio; buildSpawnOptions also forwards options.shell. This is a host process-execution boundary controlled by consumer command construction.
- Filesystem KeyValueStore rejects empty, dot, and dot-dot keys and percent-encodes filenames, but its clear operation recursively removes the entire supplied directory; the API documentation explicitly forbids sharing that directory with unrelated data.
- Node SQLite operations are synchronous and busy_timeout can block the event loop; the default is five seconds. The adapter also exposes caller-selected backup destinations and native extension loading, temporarily enabling extension support, while executeStream deliberately dies as unimplemented.
- Hypothesis: concurrent SchemaStore.modify calls can lose updates because modify performs get, an application transform, and set without a transaction or lock in this wrapper. Backend-specific or caller synchronization must be examined before treating it as atomic. (inferred)
- Hypothesis: static-file symlinks may escape the configured root because resolveFilePath enforces lexical normalized-path containment, then serving uses filesystem.stat on that path. Realpath containment is not established by the inspected functions; exploitability depends on filesystem behavior and writable tree contents. (inferred)
- Schema JIT uses globalThis.Function and a module-level compiler registry. Blocked dynamic construction or compilation failures retain interpretation, but previously captured parser entries are not replaced, making installation order an explicit behavioral constraint.
- Privileged CI deserves separate review: release-queue uses pull_request_target, checks out PRs after its fork environment gate, and grants write permissions; snapshot also gates forks. Bundle-comment consumes untrusted artifacts with symlink, size, line-count, and format validation before writing comments.
Extension points
Extension primarily happens through typed service implementations and declarative builders. HTTP groups, RPC tags, command trees, SQL drivers, AI providers, and workflow handlers each define explicit registration contracts. Schema compilation also has a shared registry, though its internal functions are not public plugin APIs. Extensions must preserve their error channels, required services, lifecycle ownership, and published export boundaries.
- Context.Service and Context.Reference define injectable keys; Layer.fromBuild provides a low-level constructor receiving MemoMap and Scope, creates a child scope, and closes it if acquisition fails.
- New HTTP handlers plug into HttpApiBuilder.group's handler builder and become routes through handlerToRoute; endpoint middleware is resolved from Context, with security middleware receiving credential-aware adapters.
- RpcGroup.make indexes definitions by _tag; add, merge, omit, and middleware compose protocols, and toHandlers registers implementation functions under each rpc.key with their captured service Context.
- The CLI command factory accepts config, contextConfig, globalFlags, aliases, examples, subcommands, custom parse functions, and a handler; runWith builds flag registries and supplies setting values as Context services before execution.
- SQL drivers plug connection acquirers, a Statement compiler, transaction controls, row transforms, and tracing attributes into SqlClient.make; transaction-aware acquisition keeps statements on the current transaction connection.
- AI providers implement LanguageModel.make's generateText and streamText hooks plus an optional codecTransformer; application tools use Toolkit.make and toLayer to register handlers, while OpenAI offers construction-time and request-context HttpClient transforms.
- The internal schema registry supports per-AST set/setCompiler and global install, caches entries in a WeakMap, and falls back to interpreted operations when compiled implementations are unavailable; global JIT enabling is a separate side-effect import.
- Workflow.make defines payload/result schemas and an idempotencyKey; workflow.toLayer delegates handler registration to WorkflowEngine.register. The in-memory engine indexes registered workflows by tag and executions by executionId, rejecting unregistered workflow execution.
Contributor handoff
Begin with tracked repository instructions, package exports, nearby implementations, and representative tests before proposing changes. The investigation used public read-only source, confirmed the supplied HEAD, and inspected tests and CI without executing them. Source observations and hypotheses should remain distinct from runtime results; this brief is bounded, not exhaustive. Small changes must respect generated files, type contracts, platform differences, and resource-finalization order. Tests, CI, and upstream code were not executed.
- Read .agents/AGENTS.md first: it documents package layout and narrow validation. scripts/setup-agents.mjs creates a root AGENTS.md symlink to that tracked file only when the root target does not already exist.
- Use the root pnpm scripts and pinned pnpm@11.20.0. Shared TypeScript configuration uses ES2022, NodeNext, explicit relative-extension rewriting, strict checking, exact optional properties, and project references; CI setup selects Node 26.4.0.
- Formatting uses two spaces, double quotes, no semicolons, and no trailing commas. Production lint rules reject barrel and .js-extension imports; root overrides relax barrel imports for tests, examples, documentation, and scripts.
- Generated barrel sections must be updated through codegen; LLMS.md is generated from ai-docs/src and migration/v3-to-v4.md from migration annotations. AI provider generation is automated by the nightly workflow, so generated/provider changes require source-generator awareness.
- For runtime lifecycle edits, read ManagedRuntime.make/dispose and its cleanup-order regressions together. For persistence edits, preserve schema encoding, missing-value semantics, binary/string discrimination, and transaction-bound connection ownership.
- Tests and CI were inspected, not executed here. For later authorized changes, repository guidance calls for targeted pnpm test --run <test_file.ts>, targeted pnpm test-types for compiler behavior, and relevant checks; bare pnpm test or doctest starts watch mode.
- Test imports can bypass public export restrictions through tsconfig.tests aliases, so successful source tests alone do not establish published import compatibility. Inspect manifest export maps and the CI build/check-dist-types path when changing public surfaces.
- Check stability annotations and synchronized-release impact before changing APIs: unstable APIs can break in minor releases and experimental APIs in patches; packages share a fixed Changesets group. Documentation-only AI changes do not require a changeset.
Languages
| Language | Files | Share of source |
|---|---|---|
| TypeScript | 1652 | 99% |
| YAML | 552 | docs/config |
| Markdown | 171 | docs/config |
| JSON | 111 | docs/config |
| JavaScript | 13 | 1% |
| Shell | 7 | 0% |
| HTML | 3 | docs/config |
Start reading here
Probable source entrypoint: packages/ — inferred from the directory tree and manifest, not the upstream docs.
Source roots
packages/ (1855 files)
Source layout — packages/
| Path | Files |
|---|---|
packages/atom/ | 29 |
packages/opentelemetry/ | 18 |
packages/tools/ | 187 |
packages/vitest/ | 6 |
Repository structure
| Top-level path | Files |
|---|---|
packages/ | 1855 |
migration/ | 552 |
ai-docs/ | 76 |
.agents/ | 31 |
.changeset/ | 17 |
scripts/ | 16 |
.github/ | 9 |
scratchpad/ | 3 |
Docs found: README.md
Package
package.json — , 46 declared dependencies.
@babel/cli@babel/core@babel/plugin-transform-export-namespace-from@babel/plugin-transform-modules-commonjs@changesets/changelog-github@changesets/cli@effect/ai-docgen@effect/bundle@effect/docgen@effect/doctest@effect/jsdocs@effect/oxc
CI & config entrypoints
README.md— Repository readme.github/workflows/release.yml— GitHub Actions workflow.github/workflows/bundle-comment.yml— GitHub Actions workflow.github/workflows/ai-codegen.yml— GitHub Actions workflow.github/workflows/snapshot.yml— GitHub Actions workflow.github/workflows/check.yml— GitHub Actions workflow.github/workflows/release-queue.yml— GitHub Actions workflow.github/workflows/cluster.yml— GitHub Actions workflowpackage.json— npm scripts: ai-docgen, ai-docgen:watch, api-diff, build, bundle-analyze, bundle-compare, bundle-compare-selected, changeset-publishpackages/effect/package.json— npm scripts: babel, build, check, codegen
Structural health signals
Average score 6.6/10 across 96 analyzed source files; 38 hotspots surfaced.
packages/effect/src/rpc/RpcServer.ts— 1508 lines, score 4.7 (Large file, Deep nesting)packages/effect/src/rpc/RpcClient.ts— 1469 lines, score 4.7 (Large file, Deep nesting)packages/effect/src/RequestResolver.ts— 1315 lines, score 4.7 (Large file, Deep nesting)
These are deterministic review leads — size, churn, and structural signals — not proof of a bug in this project.
Recent history
6389d9ac64c02026-10-02 — Fix OpenAPI client inference for possibly undefined response configuration (#8672)b9188c67ae902026-10-02 — Fix OpenAPI client includeResponse inference (#8669)20dcd50605812026-10-02 — fix: preserve stateful MCP extension capabilities (#8664)04706eff5da92026-10-02 — fix(schema): canonicalize imported JSON Schema pattern sources (#8657)86c8fb1287222026-10-02 — Avoid busy polling in Sharding reset race test (#8665)