# Extension points — ggml-org/llama.cpp

> Canonical page: https://codevetter.com/unpack/llama-cpp/extension-points

# Extension points — ggml-org/llama.cpp

Scanned commit 7ab4ee7baad2d920464cbacfad4f4b07cf111fd2 (2026-09-22). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Primary extension seams: new model architectures (conversion/ ModelBase.register + src/models/*.cpp + llama_model_mapping switch + gguf constants), new ggml backends (ggml_backend_reg_t + dlopen registry), new samplers (llama_sampler interface + common_sampler_init chain), new chat/tool parsers (common/parsers + PEG/jinja), new CLI args (common_arg table), new server endpoints (ctx_http.get/post + server_routes handlers), new built-in tools (server_tool subclasses), and new unified-binary subcommands (cmds[] table).
- New model architecture recipe is documented: register a TextModel/MmprojModel subclass via @ModelBase.register in conversion/, add MODEL_ARCH + MODEL_TENSORS entries in gguf-py/gguf/constants.py, add LLM_ARCH_* enum, a src/models/<arch>.cpp graph builder, and a case in llama_model_mapping — see docs/development/HOWTO-add-model.md. — [docs/development/HOWTO-add-model.md](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/docs/development/HOWTO-add-model.md) [src/llama-model.cpp#L43-373](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-model.cpp#L43-L373) [gguf-py/gguf/constants.py](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/gguf-py/gguf/constants.py) [conversion/base.py](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/conversion/base.py)
- ggml backends are plugins: a backend exports a ggml_backend_*_reg() registered statically under #ifdef GGML_USE_<X> or loaded at runtime by ggml_backend_load(path) which dlopens a shared library — GGML_BACKEND_DL builds each backend as a separate .so. — [ggml/src/ggml-backend-reg.cpp#L115-200](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/ggml/src/ggml-backend-reg.cpp#L115-L200) [ggml/src/ggml-backend-reg.cpp#L393](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/ggml/src/ggml-backend-reg.cpp#L393) [ggml/src/ggml-backend-dl.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/ggml/src/ggml-backend-dl.cpp)
- Samplers implement the llama_sampler interface (src/llama-sampler.cpp provides ~20 stock samplers); common_sampler_init assembles them into a chain in a data-driven order, so adding a sampler means a new llama_sampler_init_* plus a slot in the ordering switch. — [src/llama-sampler.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-sampler.cpp) [common/sampling.cpp#L187-404](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/sampling.cpp#L187-L404)
- Chat/tool-call parsing is extensible at three layers: the standalone jinja engine (common/jinja) for templates, per-model PEG parsers in common/parsers/*.cpp (one file per model family), and chat-auto-parser which auto-detects model-specific features — all surfaced via common_chat_templates_apply/common_chat_parse. — [common/chat.h#L316-340](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/chat.h#L316-L340) [common/parsers/kimi-k2.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/parsers/kimi-k2.cpp) [common/chat-auto-parser.h](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/chat-auto-parser.h) [common/jinja/README.md](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/jinja/README.md)
- CLI arguments are declarative: each option is a common_arg entry in common_params_parser_init with flags, env var (set_env), applicable examples (set_examples/set_excludes), and handler lambda; get_common_arg_defs exposes the table for introspection. — [common/arg.cpp#L77-141](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/arg.cpp#L77-L141) [common/arg.cpp#L3379-3420](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/arg.cpp#L3379-L3420)
- Server endpoints plug in via ctx_http.get/post/del + a handler on server_routes (routes registered in llama_server); all handlers are wrapped by ex_wrapper which maps std::invalid_argument to 400 and other exceptions to 500 with an OAI-style error body. — [tools/server/server.cpp#L54-86](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L54-L86) [tools/server/server.cpp#L248-306](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L248-L306)
- Built-in server tools extend the server_tool interface (name/schema/run) and are appended in the tools registry (tools.push_back(std::make_unique<server_tool_...>())); an isolate-backed tools_io abstracts file/exec ops for sandboxed runtimes. — [tools/server/server-tools.cpp#L149-207](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-tools.cpp#L149-L207) [tools/server/server-tools.cpp#L516-624](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-tools.cpp#L516-L624) [tools/server/server-tools.cpp#L1960](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-tools.cpp#L1960)
- New unified-binary subcommands: add an entry to the cmds[] table in app/llama.cpp (name, desc, aliases, hidden flag, func) — hidden commands exist but aren't listed in default help. — [app/llama.cpp#L49-78](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/app/llama.cpp#L49-L78)
- KV/memory is abstracted behind llama_memory_i (seq_rm/seq_cp/seq_keep/get_can_shift/apply); multiple implementations already coexist (llama-kv-cache, -iswa, -dsa, -dsv4, -msa, memory-hybrid, memory-recurrent) selected per architecture. — [src/llama-memory.h#L73-129](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-memory.h#L73-L129) [src/llama-kv-cache.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-kv-cache.cpp) [src/llama-memory-recurrent.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-memory-recurrent.cpp)
- Structured output plugs through grammars: standalone .gbnf files in grammars/, JSON-Schema->GBNF conversion, and an optional llguidance backend (LLAMA_LLGUIDANCE) via llama_sampler_init_llg in the sampler chain. — [grammars/README.md](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/grammars/README.md) [common/json-schema-to-grammar.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/json-schema-to-grammar.cpp) [common/sampling.cpp#L215](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/sampling.cpp#L215)
- Speculative decoding types are enumerable and extensible: common_speculative_type_from_name / _from_gguf map names and GGUF metadata to enum common_speculative_type (draft, MTP, ...), implemented in common/speculative.cpp (~3k lines). — [common/speculative.h#L8-40](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/speculative.h#L8-L40) [common/speculative.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/speculative.cpp)
- MCP servers extend tool surface without code changes: server_mcp_server_config::parse_from_json accepts Cursor-style 'mcpServers' JSON to spawn stdio MCP servers whose tools are listed via list_tools and invoked via call_tool. — [tools/server/server-mcp.cpp#L132-176](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-mcp.cpp#L132-L176) [tools/server/server-mcp.cpp#L283-334](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-mcp.cpp#L283-L334)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)