CodeVetter · Repo Unpack · Testing signals
denoland/deno
Testing is Rust-driven but mostly JS/JSONC-authored. Five separate cargo test crates each use `file_test_runner` with `harness = false` and a custom `main()`: `specs_tests` (CLI integration via `__test__.jsonc`), `unit_tests` (`tests/unit/*_test.ts`), `unit_node_tests`, `node_compat_tests`, and `integration_tests` (28 Rust files). Web Platform Tests run from a git submodule under `tests/wpt`, and results are uploaded to wpt.fyi and S3 from CI. Each runner computes a content hash of its inputs (`check_ci_hash`) and skips entirely when nothing changed. Coverage gaps are structural rather than accidental: the spec suite exercises the CLI as a black box, so internal Rust units like `CliFactory` wiring, the tunnel client, and `standalone/binary.rs` framing have little direct unit coverage.
- Spec tests are the primary integration suite: `tests/specs/mod.rs::main` collects one test per directory containing `__test__.jsonc` via `TestPerDirectoryCollectionStrategy`, wrapped in a `FileTestMapperStrategy` that expands the manifest's `tests` map into individual cases.
- There are 58 top-level spec categories under `tests/specs/` (add, bundle, cert, check, compile, coverage, cpu_prof, deploy, jsr, jupyter, lockfile, npm, permission, publish, serve, task, watch, workspaces, x, …), each containing many per-directory manifests.
- The spec runner explicitly blocks agent misuse: if `CLAUDE_CODE_ENTRYPOINT` is set and no filter argument was passed, `main()` panics with 'Running the entire spec test suite from Claude Code is not allowed. Please provide a test filter'.
- Every runner short-circuits on unchanged inputs via `test_util::hash::check_ci_hash`, which hashes `tests/specs`, `tests/util`, `tests/testdata`, `tests/registry`, the `deno` binary, `test_server`, and `denort`; a `CiHashStatus::Skip` returns immediately.
- Unit tests are TS files run through cargo: `unit_tests` collects `tests/unit/.*_test\.ts$` with `TestPerFileCollectionStrategy`; `CLAUDE.md` warns they must not be run directly with `./target/debug/deno test` because they depend on the cargo harness setup.
- Flakiness is first-class: the spec runner threads a shared `FlakyTestTracker` through `run_maybe_flaky_test`, and the `__test__.jsonc` schema has a `flaky: boolean` field plus an `if` field for platform gating (`mac`, `linux`, `windows`, `unix`, `notMacIntel`).
- Spec tests run against a live local HTTP server: `let _http_guard = test_util::http_server();` is held for the duration of the run, and `tests/util/server/` builds the `test_server` binary with `servers/`, `https.rs`, and `npm.rs` fixtures.
- Rust-side integration tests live in `tests/integration/` as 28 files (`lsp_tests.rs`, `npm_tests.rs`, `publish_tests.rs`, `watcher_tests.rs`, `upgrade_tests.rs`, `compile_tests.rs`, `sqlite_extension_tests.rs`, …) in the `integration_tests` crate, with an `upgrade` cargo feature gating upgrade tests.
- Node.js compatibility is tested against the real Node test suite pulled in as a submodule: CI clones `./tests/node_compat/runner/suite` only for the `node_compat` test crate, and the results are summarised by `tests/node_compat/report.rs` and `slack.ts`.
- WPT runs on non-tag builds: CI clones `tests/wpt/suite`, writes the hosts file, runs `./tests/wpt/wpt.ts run --all --quiet`, and on release uploads `wpt.json`/`wptreport.json.gz` to `s3://dl-deno-land/wpt/` and to wpt.fyi via `tools/upload_wptfyi.js`.
- Core crate tests run under cargo-nextest, not plain cargo test: `./x test-core` and CI both run `cargo nextest run` across `deno_core`, `deno_v8`, `build-your-own-js-snapshot`, `dcore`, `deno_ops`, `deno_ops_compile_test_runner`, `serde_v8`, `deno_core_testing`, plus a separate `cargo test --doc` pass.
- CI asserts tests do not mutate the working tree: after running a test crate it checks git status and fails with '❌ Git working directory is dirty. Ensure `cargo test` is not modifying git tracked files.'
- Lint is itself partly a test suite: `tools/lint.js` runs clippy plus bespoke checks — `ensureNoNonPermissionCapitalLetterShortFlags`, `ensureDisallowedMethodsEnforced`, `lintBootstraps`, `lintNodePolyfillDenoApis`, `ensureWorkflowYmlsUpToDate`, `ensureNoUnusedOutFiles`, `ensureNoNewTopLevelEntries`, and `checkCopyright`.
- Test sharding and parallelism are built into the harness: `ShardConfig::from_env()` filters the collected category, and `Parallelism::default().max_parallelism()` drives `file_test_runner::run_tests`.
- The tunnel client, `initialize_tunnel`'s credential fallback chain, and `cli/standalone/binary.rs` binary framing have no dedicated test module in their files — unlike `cli/tools/deploy.rs`, `cli/worker.rs`, and `cli/file_fetcher.rs`, which each carry `#[cfg(test)] mod tests`. (inferred)