CodeVetter · Repo Unpack · Contributor handoff
ggml-org/llama.cpp
Read AGENTS.md first — it is strict: ASCII-only, concise comments, no new test files without maintainer approval, never commit/push/PR on the user's behalf, reuse existing infra. Build with cmake -B build && cmake --build build -j; run C++ tests via ctest and server tests via tools/server/tests/tests.sh. Key files to read before editing: common/arg.cpp (params), tools/server/README-dev.md (server feature scope), docs/development/HOWTO-add-model.md (arch additions), and the pimpl of whatever you touch (server_context_impl, llama_model subclasses).
- AGENTS.md is mandatory and unusual: it forbids agents from writing PR descriptions/comments/reviewer responses, forbids git push/gh pr create entirely (project ban risk), requires 'Assisted-by:' not 'Co-authored-by:' in human-approved commits, and tells fully autonomous agents to stop and exclude llama.cpp from contribution targets.
- Style rules enforced by maintainers: no emdashes/unicode arrows (use ASCII '-', '->', 'x', '...'), comments are 1-2 lines max in ASD-STE100 simplified English, no hard-wrapping comments, no comment-first-then-code, and do NOT split a line mid-sentence to fit a column width.
- Do not add new files under tests/ without maintainer approval — reuse existing test infrastructure; test-chat links against the server-context library so server chat changes can be exercised from tests/test-chat.cpp.
- Server feature scope is contractually bounded by tools/server/README-dev.md: in-scope = inference, chat/tool calling, API compat, multimodal, memory mgmt, Web UI needs; out-of-scope = server-side agentic loops, internal-state APIs, model-specific features; file-IO features (MCP, model save/load) must be disabled by default for security.
- Build: cmake -B build (Release default, BUILD_SHARED_LIBS on) then cmake --build build -j produces bin/llama plus llama-* tools; LLAMA_BUILD_UI embeds the web UI; LLAMA_SUBPROCESS is off on iOS/Android/Emscripten which disables server router mode and server tools.
- Test invocation: ctest --test-dir build (label 'main' runs the core suite; 'model' needs downloaded models); server suite: cd tools/server/tests && ./tests.sh (needs PYTEST deps, spawns built server); UI: cd tools/ui && npm test (vitest+playwright).
- Before touching CLI behavior read common/arg.cpp's common_params_parser_init — every flag is a common_arg with examples/env/handler; adding a flag means one table entry plus documentation regeneration; LLAMA_EXAMPLE_* gating controls which binaries accept the flag.
- Server edits: the inference core is server_context_impl inside tools/server/server-context.cpp (pimpl over server_context); slot scheduling lives in process_single_task/get_available_slot/launch_slot_with_task; decode loop is update_slots -> pre_decode/decode/post_decode; HTTP routes are registered in server.cpp not in server-context.
- Adding a model arch requires coordinated edits across five places: conversion/<arch>.py (ModelBase.register), gguf-py/gguf/constants.py (MODEL_ARCH/MODEL_TENSORS), src/llama-arch.* (LLM_ARCH_*), src/models/<arch>.cpp (graph builder), and llama_model_mapping in src/llama-model.cpp — HOWTO-add-model.md is the canonical guide.
- The chat stack has traps: llama.cpp uses its own Jinja engine in common/jinja (NOT minja — AGENTS.md explicitly warns about stale knowledge); tool-call parsers are PEG-based in common/peg-parser.cpp + common/parsers; use test-chat-peg-parser/test-jinja to validate changes.
- Concurrency model: a single decode thread runs queue_tasks.start_loop -> update_slots; HTTP handlers run on httplib threads and only communicate via queue_tasks.post + queue_results; decode yields via yield_to_queue only for metrics/slot-read tasks — never call llama_decode off the main loop.
- Known foot-guns: GGML_ABORT crashes on error paths; llama_decode errors clear entire slots; llama-server.cpp default port 8080 is slated to change to 9931; embeddings require n_batch<=n_ubatch (auto-clamped with warning); 'completion' is the legacy hidden tool — prefer server/cli paths for new work.