# Feature catalog — ggml-org/llama.cpp

> Canonical page: https://codevetter.com/unpack/llama-cpp/feature-catalog

# Feature catalog — ggml-org/llama.cpp

Scanned commit 7ab4ee7baad2d920464cbacfad4f4b07cf111fd2 (2026-09-22). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

User-facing surface splits into the `llama` unified binary subcommands, the llama-server HTTP API, the interactive `llama cli` chat client, offline analysis/conversion tools, and the embedded web UI. The server exposes OpenAI-compatible, Anthropic-compatible, Ollama-ish and legacy endpoints plus extras (rerank, infill, tokenize, slots save/load, LoRA hotswap, resumable streams, MCP tools). Feature scope for the server is deliberately bounded by tools/server/README-dev.md.
- llama-server registers a broad REST surface in llama_server(): /completion(s), /v1/completions, /chat/completions + /v1/chat/completions, /v1/responses, /v1/audio/transcriptions, /v1/messages + /v1/messages/count_tokens (Anthropic), /infill, /embedding(s) + /v1/embeddings, /rerank(ing) + /v1/rerank(ing), /tokenize, /detokenize, /apply-template, /chat/completions/input_tokens, /props, /models, /metrics, /health, /lora-adapters (GET/POST), /slots + /slots/:id_slot, and resumable-stream endpoints /v1/stream and /v1/streams/lookup. — [tools/server/server.cpp#L248-306](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L248-L306)
- Router mode activates when no model is specified: llama_server() rewires most route handlers to models_routes->proxy_post/proxy_get and adds /models/load, /models/unload, /models/sse and DELETE /models so models are loaded on demand into child processes. — [tools/server/server.cpp#L136-246](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L136-L246) [tools/server/server-models.cpp#L255-367](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-models.cpp#L255-L367)
- `llama cli` provides an interactive REPL with slash commands /exit, /regen, /clear, /read <file>, /glob <pattern>, /image, /audio, /video, a banner showing model/ftype/modalities, multimodal staging via base64 data URLs, and optional --out-file logging of the session. — [tools/cli/cli-context.cpp#L413-456](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/cli/cli-context.cpp#L413-L456) [tools/cli/cli-context.cpp#L517-555](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/cli/cli-context.cpp#L517-L555) [tools/cli/cli-context.cpp#L306-341](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/cli/cli-context.cpp#L306-L341)
- Hidden CLI subcommands still ship in the binary: completion (text completion), bench, batched-bench, fit-params (auto-size params to fit device memory), quantize, and perplexity — all marked hidden=true in the cmds[] table. — [app/llama.cpp#L64-78](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/app/llama.cpp#L64-L78) [tools/completion/completion.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/completion/completion.cpp) [tools/llama-bench/llama-bench.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/llama-bench/llama-bench.cpp) [tools/perplexity/perplexity.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/perplexity/perplexity.cpp)
- Speculative decoding is a first-class feature via common/speculative.cpp: draft-model, MTP (multi-token prediction) and other types are parsed by common_speculative_types_from_names / inferred from GGUF metadata, and the server tracks draft-acceptance metrics per slot. — [common/speculative.h#L8-30](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/speculative.h#L8-L30) [common/speculative.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/speculative.cpp) [tools/server/server-context.cpp#L4137-4139](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L4137-L4139)
- Structured output is supported through GBNF grammars (sample grammars in grammars/), json-schema-to-grammar.cpp converting JSON Schema to GBNF, and optional llguidance integration (LLAMA_LLGUIDANCE cmake option, llama_sampler_init_llg). — [common/json-schema-to-grammar.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/json-schema-to-grammar.cpp) [common/sampling.cpp#L215](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/sampling.cpp#L215) [CMakeLists.txt#L146](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/CMakeLists.txt#L146) [grammars/json.gbnf](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/grammars/json.gbnf)
- Chat formatting/parsing is handled by a dedicated Jinja engine (common/jinja, lexer+parser+runtime) plus a PEG-based parser framework (common/peg-parser.cpp, chat-peg-parser.cpp) and per-model tool-call parsers in common/parsers/ (kimi-k2, gpt-oss, ministral3, cohere2moe, etc.); common_chat_templates_init/apply produce the prompt and common_chat_parse extracts reasoning/tool calls. — [common/jinja/README.md](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/jinja/README.md) [common/chat.h#L316-340](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/chat.h#L316-L340) [common/parsers/kimi-k2.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/parsers/kimi-k2.cpp) [common/chat-peg-parser.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/chat-peg-parser.cpp)
- Model acquisition features: -hf <repo> Hugging Face downloads with an ETag-based file cache (hf-cache.cpp, write_etag/read_etag, atomic .tmp+rename writes), --model-url direct download, docker_repo support, and HF remote preset repos applied in router mode. — [common/download.cpp#L50-90](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/download.cpp#L50-L90) [common/hf-cache.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/hf-cache.cpp) [tools/server/server.cpp#L122-134](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L122-L134)
- Server-side agent features: --tools enables built-in tools including exec_shell_command, file read/write tools, and MCP server integration (--mcp-*) where MCP servers run as stdio subprocesses via common_subproc; disabled features return a 403 'feature_disabled' response. — [tools/server/server-tools.cpp#L1239-1247](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-tools.cpp#L1239-L1247) [tools/server/server-tools.cpp#L1960](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-tools.cpp#L1960) [tools/server/server-mcp.cpp#L340-351](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-mcp.cpp#L340-L351) [tools/server/server.cpp#L312-371](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L312-L371)
- KV/memory management features: multi-slot continuous batching with n_parallel slots, unified KV cache (--kv-unified / --kv-unified-per-slot auto-sizing), prompt cache in RAM (--cache-ram / cache_idle_slots), context checkpoints (--ctx-checkpoints), context shifting, and slot save/restore endpoints. — [tools/server/server.cpp#L162-172](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L162-L172) [tools/server/server-context.cpp#L1351-1370](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L1351-L1370) [tools/server/server-context.cpp#L3625-3635](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L3625-L3635)
- LoRA adapters are loaded per request and hot-swappable: common_init_from_params loads params.lora_adapters via llama_adapter_lora_init, and the server exposes GET/POST /lora-adapters plus per-batch application via common_set_adapter_lora in update_slots(). — [common/common.cpp#L1343-1358](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/common.cpp#L1343-L1358) [tools/server/server.cpp#L284-285](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L284-L285) [tools/server/server-context.cpp#L2848-2865](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L2848-L2865) [src/llama-adapter.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-adapter.cpp)
- Quantization and analysis tools: llama-quantize (tools/quantize), llama-imatrix (importance matrix), llama-perplexity (PPL/KLD eval), llama-gguf-split, llama-tokenize, llama-tts, llama-export-lora, llama-cvector-generator, and rpc-server exposing a remote compute backend. — [tools/quantize/quantize.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/quantize/quantize.cpp) [tools/imatrix/imatrix.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/imatrix/imatrix.cpp) [tools/rpc/rpc-server.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/rpc/rpc-server.cpp) [tools/gguf-split](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/gguf-split) [tools/tokenize](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/tokenize)
- Google Cloud Vertex AI compat: gcp_params in server-http.cpp reads AIP_MODE/AIP_HEALTH_ROUTE/AIP_PREDICT_ROUTE/AIP_HTTP_PORT env vars and remaps health/predict routes and port when AIP_MODE=PREDICTION. — [tools/server/server-http.cpp#L60-103](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-http.cpp#L60-L103) [tools/server/server.cpp#L308-309](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L308-L309)
- The web UI is a full chat frontend (SvelteKit static build embedded in the binary) with model management, MCP support, PWA assets, storybook stories, and its own Playwright e2e + Vitest suites configured in tools/ui/package.json. — [tools/ui/package.json#L6-27](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/ui/package.json#L6-L27) [tools/ui/vite.config.ts](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/ui/vite.config.ts) [tools/ui/playwright.config.ts](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/ui/playwright.config.ts)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)