# Testing signals — ggml-org/llama.cpp

> Canonical page: https://codevetter.com/unpack/llama-cpp/testing-signals

# Testing signals — ggml-org/llama.cpp

Scanned commit 7ab4ee7baad2d920464cbacfad4f4b07cf111fd2 (2026-09-22). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

C++ tests live in tests/ and register with CTest via llama_build_and_test/llama_test helpers; coverage spans tokenizers (against models/ggml-vocab-*.gguf fixtures), grammars, chat templates/parsers, KV/recurrent state, samplers, quantization, and backend ops. The server has a parallel pytest suite (tools/server/tests) that spawns real llama-server processes, and the web UI has Vitest (unit/client/ui projects) plus Playwright e2e. CI runs ctest -L main and dedicated per-backend workflows.
- Test registration is CMake-driven: tests/CMakeLists.txt defines llama_build, llama_test, llama_test_cmd, llama_build_and_test helpers; most tests link llama-common (some also libllama) and are labeled 'main' or 'model' for ctest -L filtering. — [tests/CMakeLists.txt#L3-117](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/CMakeLists.txt#L3-L117)
- Tokenizer coverage is fixture-heavy: test-tokenizer-0 is built once and instantiated ~15 times against models/ggml-vocab-*.gguf files (bert-bge, command-r, deepseek-coder, gemma-4, gpt-2, llama-bpe/spm, qwen2, qwen35, ...); several slower tokenizer tests are commented out. — [tests/CMakeLists.txt#L120-191](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/CMakeLists.txt#L120-L191) [models/](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/models/)
- Several tests are gated: Windows+shared-libs excludes tests using non-exported internals (test-unicode, test-sampling, test-grammar-*, test-chat, ...); test-opt is skipped under ASAN or GGML_SCHED_NO_REALLOC; backend-direct tests (test-barrier, test-quantize-*, test-rope) need non-DL CPU backend. — [tests/CMakeLists.txt#L154-172](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/CMakeLists.txt#L154-L172) [tests/CMakeLists.txt#L304-332](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/CMakeLists.txt#L304-L332)
- A dummy-model fixture system exists: test-llama-archs generates tiny GGUFs for every architecture (test-generate-models ctest fixture), then test-recurrent-state-rollback and test-save-load-state run against generated qwen35-dense, nemotron_h-dense, deepseek4-moe, kimi-k3-moe models. — [tests/CMakeLists.txt#L195-256](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/CMakeLists.txt#L195-L256) [tests/test-llama-archs.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/test-llama-archs.cpp)
- A real model is downloaded in CI: test-download-model fetches tinyllamas/stories15M-q4_0.gguf (SHA256-pinned) via cmake/download-models.cmake as a ctest fixture required by test-thread-safety and test-state-restore-fragmented. — [tests/CMakeLists.txt#L279-319](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/CMakeLists.txt#L279-L319)
- Chat/template stack has dedicated tests: test-chat (links server-context lib), test-chat-template, test-chat-peg-parser, test-chat-auto-parser, test-jinja (plus a -py labeled variant), test-json-schema(-to-grammar), test-peg-parser with a suite of peg-parser/test-*.cpp files; test-chat-analysis is built but deliberately not registered (manual debug tool). — [tests/CMakeLists.txt#L163-166](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/CMakeLists.txt#L163-L166) [tests/CMakeLists.txt#L258-277](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/CMakeLists.txt#L258-L277)
- Server has a pytest suite in tools/server/tests: tests.sh runs pytest -n auto --dist=worksteal with a 'slow' marker; utils.py ServerProcess spawns the real llama-server binary on a port with a tiny stories260K model; 27 unit files cover chat completion, OAI/Anthropic/GCP compat, embeddings, rerank, infill, lora, mcp, router, security, sleep, slots, speculative, streaming, vision, tool calls. — [tools/server/tests/tests.sh](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/tests/tests.sh) [tools/server/tests/utils.py#L38-60](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/tests/utils.py#L38-L60) [tools/server/tests/unit/test_security.py](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/tests/unit/test_security.py)
- Security tests exist for API keys: test_security.py asserts /health is public, static UI assets bypass auth, wrong/missing Bearer keys get 401 authentication_error, and a valid key returns 200. — [tools/server/tests/unit/test_security.py](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/tests/unit/test_security.py)
- UI testing is three-tiered: vitest projects 'unit', 'client' (browser mode via playwright provider) and 'ui', plus 'test:e2e' playwright tests and storybook; scripts in tools/ui/package.json wire them into npm test. — [tools/ui/package.json#L17-24](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/ui/package.json#L17-L24) [tools/ui/vite.config.ts](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/ui/vite.config.ts) [tools/ui/playwright.config.ts](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/ui/playwright.config.ts)
- CI coverage: .github/workflows/server.yml runs the server pytest suite on PRs touching C/C++ or tools/server; build-cpu.yml runs 'ctest -L main --verbose --timeout 900'; separate workflows cover CUDA/Metal/Vulkan/SYCL/sanitizers/server-sanitize; python-lint and python-type-check gate the conversion code. — [.github/workflows/server.yml#L1-40](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/.github/workflows/server.yml#L1-L40) [.github/workflows/build-cpu.yml#L127](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/.github/workflows/build-cpu.yml#L127) [.github/workflows/server-sanitize.yml](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/.github/workflows/server-sanitize.yml) [.github/workflows/python-lint.yml](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/.github/workflows/python-lint.yml)
- Coverage gaps: test-backend-ops and test-tokenizer-1-bpe exist as build targets but have most of their test registrations commented out ('disabled due to slowness'); test-double-float is commented as SLOW; ggml backend compute correctness relies heavily on test-backend-ops which is built but only selectively registered. — [tests/CMakeLists.txt#L177-193](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/CMakeLists.txt#L177-L193) [tests/CMakeLists.txt#L310](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tests/CMakeLists.txt#L310)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)