CodeVetter · Repo Unpack · Testing signals
oven-sh/bun
Testing is dominated by 2,149 JS/TS test files run through Bun's own Jest-compatible runner, orchestrated by a Node.js script (`scripts/runner.node.ts`, 3,504 lines) that deliberately avoids Bun APIs and imports nothing, spawning each file in a separate process so crashes are attributable. Coverage is heaviest in `test/js` (1,307 files) and `test/regression` (426 issue-numbered files); `test/bundler` uses a dedicated `itBundled` harness, `test/bake` covers HMR, and `test/internal` tests the build scripts themselves. Rust has thin unit coverage — 76 files contain `#[test]` out of ~700 Rust source files. CI shards the suite 20 ways on Linux with ASAN options set, and non-binary lanes (types, source lints) are excluded from Buildkite and run in GitHub Actions instead.
- There are 2,149 test files under `test/`, distributed as `test/js` 1,307, `test/regression` 426, `test/cli` 202, `test/bundler` 110, `test/napi` 60, `test/internal` 57, `test/integration` 28, `test/bake` 27, and `test/v8` 1.
- `scripts/runner.node.ts` is explicitly documented as running each `bun test` invocation in a separate process to catch crashes, as unable to use Bun APIs (it runs under Node), and as importing no dependencies so it starts fast.
- `test/harness.ts` (2,367 lines) is the shared fixture layer, exporting `bunExe()`, `bunEnv`, `tempDir`/`tempDirWithFiles`, `normalizeBunSnapshot`, platform predicates (`isMacOS`, `isMusl`, `isASAN`, `isDebug`), `describeWithContainer` for Docker-backed suites, `getFDCount`, and RSS-delta assertions for leak tests.
- Bundler tests use a purpose-built `itBundled` helper in `test/bundler/expectBundled.ts` rather than raw spawns, and the repo documents it as the required entry point for bundler/transpiler tests.
- Regression tests are namespaced by real GitHub issue number under `test/regression/issue/`, and CLAUDE.md restricts that directory to true regressions — behavior that worked in a prior release — with everything else going into the existing module test file.
- `test/internal/` tests the build system itself rather than the runtime — `build-rust-toolchain-probe.test.ts`, `build-post-link-ordering.test.ts`, `build-timings.test.ts`, `rust-check-all.test.ts`, `runner-junit.test.ts`, `parallel-allowlist.test.ts`.
- Rust unit-test coverage is sparse relative to the surface: only 76 files under `src/` contain a `#[test]` attribute and 79 contain a `#[cfg(test)]` module, against ~701 Rust source files — correctness is overwhelmingly asserted from the JS side against a built binary. (inferred)
- CI test steps are sharded by platform — `parallelism` 20 on Linux, 8 on Windows, 2 on macOS, and 1 on the beta tier — with timeouts of 30/45/60 minutes and `soft_fail` only on the beta lane.
- The Buildkite test step explicitly excludes `integration/bun-types` and `internal/source-lints`, delegating them to `.github/workflows/bun-types.yml` and `.github/workflows/source-lints.yml` because they never touch the built binary.
- CI test jobs set `ASAN_OPTIONS=allow_user_segv_handler=1:disable_coredump=0:detect_leaks=0` and assert the agent's actual platform via `EXPECTED_PLATFORM_OS`/`ARCH`/`ABI`/`DISTRO`/`RELEASE` before running any test, so a mislabeled agent fails loudly instead of silently testing the wrong target.
- Leak testing has dedicated infrastructure: a `testleak` npm script that enables `BUN_DESTRUCT_VM_ON_EXIT=1` with LSAN suppressions from `test/leaksan.supp`, plus in-binary suppressions compiled into `__lsan_default_suppressions` for Rust-side structural allocations.
- Flakiness and timing are managed as tracked data files rather than ad-hoc retries: `test/flaky-tests.txt`, `test/expected-durations.json`, `test/parallel-allowlist.json`, `test/parallel-denylist.txt`, and `test/expectations.txt`.
- The test workspace pulls in ~100 real npm packages (express, fastify, nest, prisma, typeorm, mongodb, socket.io, webpack, astro, svelte, verdaccio, vitest) so compatibility is asserted against actual ecosystem code rather than mocks.
- The documented verification contract for a new test is dual-run: it must fail under `USE_SYSTEM_BUN=1 bun test <file>` and pass under `bun bd test <file>`, otherwise the test is considered invalid.
- Test-failure triage is tooled: `bun run ci:errors`, `ci:status`, `ci:logs`, `ci:watch`, and `ci:slowest` wrap the BuildKite CLI through `scripts/find-build.ts` and `scripts/ci-slowest-tests.ts`.