CodeVetter · Repo Unpack · Testing signals
sqlite/sqlite
Testing is TCL-first: 1,195 `.test` scripts in `test/` are executed by `testfixture`, an augmented tclsh built from `src/tclsqlite.c` plus 44 `src/test*.c` harness modules that expose internals (malloc fault injection, VFS shims, thread control, syscall interception) to TCL. `test/testrunner.tcl` parallelizes across cores and drives named suites and permutations; `permutations.test` defines 58 test_suite configurations. `make devtest` — the developers' minimum pre-check-in bar — chains `srctree-check` (generated files up to date) and `sourcetest` before running `testrunner.tcl mdevtest`. Fuzzing is a separate, substantial tier: `fuzzcheck` replays fuzzer corpora stored as SQLite databases, and `ossfuzz.c` is the OSS-Fuzz entry point. No CI configuration lives in the repo; extension suites are colocated with their extensions.
- The test corpus is 1,195 `.test` TCL scripts under `test/`, run by the `testfixture` binary built from `$(TESTFIXTURE_SRC)`; `make smoketest` runs just `./testfixture test/main.test` as the fast sanity check.
- `make devtest` depends on `srctree-check` and `sourcetest`, then runs `test/testrunner.tcl mdevtest`; `test`, `quicktest` and the legacy `fulltest`/`alltest`/`soaktest` targets are aliases onto `devtest` or `releasetest`.
- `tool/srctree-check.tcl` is a guard against stale generated artifacts: it regenerates `autoconf/Makefile.msc` via `tool/mkmsvcmin.tcl` and diffs it, and checks `VERSION` against `autoconf/tea/configure.ac`, exiting non-zero with a suggested fix command.
- `test/testrunner.tcl` bootstraps its own interpreter: `find_interpreter` tries `package require sqlite3`, falls back to sourcing a local `pkgIndex.tcl`, then to re-execing under `./testfixture`, and finally offers to build the TCL extension — which is why `make tclextension-install` is recommended before testing.
- The runner estimates work per job class with weights `tcl=1`, `fuzz=22`, `bld=66`, `make=102` (optionally overridden by `test/testrunner_estwork.tcl`) so parallel scheduling packs long builds first.
- `test/testrunner_data.tcl` is the declarative test matrix: `trd_builds`, `trd_configs`, `trd_extras`, `trd_all_configs`, `trd_fuzztest_data` and `trd_buildscript` define per-platform build configurations and which suites each one runs, mirroring the old `all.test`.
- `test/permutations.test` defines 58 `test_suite` permutations, letting the same `.test` files be re-run under different pragma/compile configurations (this is also what `make valgrindtest` invokes with the `valgrind` permutation).
- 44 `src/test*.c` modules expose internals to TCL for white-box testing, including `test_malloc.c` (allocation fault injection), `test_vfs.c`/`test_devsym.c`/`test_demovfs.c` (VFS simulation), `test_journal.c`, `test_syscall.c` (syscall interception), `test_thread.c`, `test_mutex.c` and `test_pcache.c`.
- `test/fuzzcheck.c` replays fuzzer output stored in SQLite databases with `db(dbid, dbcontent)` and `xsql(sqlid, sqltext)` tables, looking only for crashes, assertion faults and leaks — it explicitly does not verify output; eight `fuzzdata*.db` corpora ship in-tree.
- `test/ossfuzz.c` is the Google OSS-Fuzz `LLVMFuzzerTestOneInput` harness with debug flags (`FUZZ_SQL_TRACE`, `FUZZ_SHOW_MAX_DELAY`, `FUZZ_SHOW_ERRORS`) that can be driven interactively through `test/ossshell.c`.
- Additional specialized fuzzers exist for distinct attack surfaces: `test/dbfuzz.c` and `test/dbfuzz2.c` (database file images), `test/sessionfuzz.c` (changeset blobs), `test/optfuzz.c` (query optimizer equivalence), and `test/fuzzinvariants.c` (result invariants).
- Extension test suites live beside their extensions rather than in `test/`: 149 files under `ext/fts5/test`, 44 `.test` files in `ext/session/`, 43 in `ext/rbu/`, 27 in `ext/rtree/`, 15 in `ext/recover/`, and 5 in `ext/intck/`.
- Multi-process concurrency is tested by a separate harness, `mptest/mptest.c` with `config01.test`, `config02.test`, `crash01.test`, `crash02.subtest` and `multiwrite01.test`, exercising cross-process locking that single-process TCL tests cannot reach.
- Failure triage is documented and tooled: `test/testrunner.tcl errors` and `status` targets are wired into `main.mk`, and AGENTS.md instructs checking `grep '!' testrunner.log`.
- No in-repo CI exists — there is no `.github/`, `.gitlab-ci.yml`, or `.travis.yml` — so coverage of the release matrix (`releasetest`, valgrind, MSVC via `tool/build-all-msvc.bat`, `tool/omittest.tcl` for OMIT-flag combinations) depends on developers running it locally.