# Behavior traces — ggml-org/llama.cpp

> Canonical page: https://codevetter.com/unpack/llama-cpp/behavior-traces

# Behavior traces — ggml-org/llama.cpp

Scanned commit 7ab4ee7baad2d920464cbacfad4f4b07cf111fd2 (2026-09-22). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Three flows traced end-to-end: (1) `llama serve` startup including router vs single-model branches, (2) an HTTP chat completion through tokenize -> task queue -> slot -> decode -> SSE, and (3) `llama cli` startup which spawns an embedded server thread and streams chat over loopback HTTP.
- Server startup trace: app main() matches 'serve' -> llama_server(argc,argv) -> setlocale/SIGPIPE ignore -> common_init() -> server_stream_session_manager_start() -> common_params_parse(LLAMA_EXAMPLE_SERVER) -> llama_backend_init() + llama_numa_init() -> llama_server(params,...) which resolves HF presets, sizes KV pool, and constructs server_mcp, server_context, server_http_context. — [app/llama.cpp#L129-143](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/app/llama.cpp#L129-L143) [tools/server/server.cpp#L88-190](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L88-L190)
- Server startup (continued): ctx_http.init(params) builds the httplib (SSL)Server -> routes registered on ctx_http with ex_wrapper -> mcp_mgr.start + tools.setup -> model download via common_models_handler_apply -> ctx_http.start() (before model load so /health works) -> ctx_server.load_model(params) -> routes.update_meta -> ctx_server.start_loop() blocks the main thread in queue_tasks.start_loop(). — [tools/server/server.cpp#L192-246](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L192-L246) [tools/server/server.cpp#L383-493](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L383-L493) [tools/server/server.cpp#L540-558](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L540-L558)
- Inside load_model -> init(): wires queue callbacks (on_new_task -> process_single_task, on_update_slots -> update_slots, on_sleeping_state -> handle_sleeping_state), inits metrics, reads LLAMA_TRACE / LLAMA_SERVER_SLOTS_DEBUG / LLAMA_SERVER_SLOTS_N_DIFF env vars, sizes the shared llama_batch to max(n_batch, n_parallel), creates the RAM prompt cache, and initializes chat templates via common_chat_templates_init. — [tools/server/server-context.cpp#L1290-1460](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L1290-L1460)
- Chat completion trace: POST /v1/chat/completions -> routes.post_chat_completions -> handle_completions_impl -> tokenize/process_mtmd_prompt -> server_task(type=COMPLETION) per input -> rd.post_tasks -> server_queue::start_loop drains queue -> process_single_task -> get_available_slot -> launch_slot_with_task -> next update_slots() iteration batches it. — [tools/server/server-context.cpp#L4257-4342](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L4257-L4342) [tools/server/server-queue.cpp#L278-316](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-queue.cpp#L278-L316) [tools/server/server-context.cpp#L2382-2453](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L2382-L2453)
- Generation loop trace: update_slots() early-returns when all slots idle (else self-posts SERVER_TASK_TYPE_NEXT_RESPONSE) -> pre_decode() applies context-shift/checkpoints -> batch.render() -> for each n_batch chunk decode() -> queue_tasks.yield_to_queue wrapping llama_decode(ctx_tgt, batch_view) + llama_synchronize -> post_decode() samples and emits partial/final results. — [tools/server/server-context.cpp#L2807-2907](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L2807-L2907) [tools/server/server-context.cpp#L3673-3686](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L3673-L3686)
- Streaming response trace: first rd.next() result decides headers; then res->set_next drives chunked output — format_oai_sse/format_anthropic_sse/format_oai_resp_sse per result, SSE keepalive ping ':' when sse_ping_interval elapses, 'data: [DONE]' terminator for OAI formats; errors mid-stream are formatted as SSE error events. — [tools/server/server-context.cpp#L4380-4523](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L4380-L4523)
- CLI startup trace: llama_cli -> common_init() -> common_params_parse(LLAMA_EXAMPLE_CLI) -> install SIGINT/SIGTERM handler -> cli_context(params) -> init(): either connect to --server-base or cli_server.start() spawns llama_server() on a std::thread with a random port, wait_ready() polls GET /health until 200, then client.wait_health and list_and_ask_models() (GET /v1/models) pick a model. — [tools/cli/cli.cpp#L33-64](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/cli/cli.cpp#L33-L64) [tools/cli/cli-server.h#L34-84](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/cli/cli-server.h#L34-L84) [tools/cli/cli-context.cpp#L94-183](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/cli/cli-context.cpp#L94-L183)
- CLI turn trace: run() prints banner -> user_turn.read_input() -> slash-command dispatch (/regen pops last message, /clear resets, /image|/audio|/video stages base64, /read and /glob inline file text) -> push_user_message -> generate_completion POSTs SSE to /v1/chat/completions -> delta.reasoning_content/delta.content rendered via ui::assistant_turn, timings pulled from chunk.timings. — [tools/cli/cli-context.cpp#L413-560](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/cli/cli-context.cpp#L413-L560) [tools/cli/cli-context.cpp#L350-411](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/cli/cli-context.cpp#L350-L411)
- Model load trace (persistence path): llama_model_load_from_file -> llama_model_load_from_file_impl validates exactly one of metadata/path/file, installs a default progress callback -> llama_model_load -> llama_model_loader ctor (gguf parse, split resolution, mmap) -> llama_model_create(ml, params) -> llama_prepare_model_devices -> model->load_hparams -> load_vocab -> load_stats -> print_info -> load_tensors (supports vocab_only early return and -2 cancellation via progress callback). — [src/llama.cpp#L316-444](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama.cpp#L316-L444) [src/llama-model-loader.cpp#L80-105](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-model-loader.cpp#L80-L105)
- Router child spawn trace: server_models::load (server-models.cpp) builds child argv, sets subprocess_option_no_window|combined_stdout_stderr, calls inst.subproc->sproc.create(child_args, options, child_env); a single waiter thread watches all children, parses 'cmd_child_to_router:state:' lines from stdout and calls on_child_exit on EOF/exit; request_child_exit writes 'cmd_router_to_child:exit' to the child's stdin. — [tools/server/server-models.cpp#L45-80](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-models.cpp#L45-L80) [tools/server/server-models.cpp#L184-255](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-models.cpp#L184-L255) [tools/server/server-models.cpp#L1163-1280](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-models.cpp#L1163-L1280)
- Shutdown trace: SIGINT -> signal_handler -> shutdown_handler -> (single-model) mcp_mgr.shutdown() + ctx_server.terminate() which unblocks start_loop -> clean_up(): server_stream_session_manager_stop, ctx_http.stop, ctx_server.terminate, llama_backend_free; a second Ctrl+C force-exits with exit(1). — [tools/server/server.cpp#L25-49](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L25-L49) [tools/server/server.cpp#L448-558](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server.cpp#L448-L558)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)