ggml-org/llama.cpp

Three flows traced end-to-end: (1) `llama serve` startup including router vs single-model branches, (2) an HTTP chat completion through tokenize -> task queue -> slot -> decode -> SSE, and (3) `llama cli` startup which spawns an embedded server thread and streams chat over loopback HTTP.

• Server startup trace: app main() matches 'serve' -> llama_server(argc,argv) -> setlocale/SIGPIPE ignore -> common_init() -> server_stream_session_manager_start() -> common_params_parse(LLAMA_EXAMPLE_SERVER) -> llama_backend_init() + llama_numa_init() -> llama_server(params,...) which resolves HF presets, sizes KV pool, and constructs server_mcp, server_context, server_http_context.
  app/llama.cpp#L129-143tools/server/server.cpp#L88-190
• Server startup (continued): ctx_http.init(params) builds the httplib (SSL)Server -> routes registered on ctx_http with ex_wrapper -> mcp_mgr.start + tools.setup -> model download via common_models_handler_apply -> ctx_http.start() (before model load so /health works) -> ctx_server.load_model(params) -> routes.update_meta -> ctx_server.start_loop() blocks the main thread in queue_tasks.start_loop().
  tools/server/server.cpp#L192-246tools/server/server.cpp#L383-493tools/server/server.cpp#L540-558
• Inside load_model -> init(): wires queue callbacks (on_new_task -> process_single_task, on_update_slots -> update_slots, on_sleeping_state -> handle_sleeping_state), inits metrics, reads LLAMA_TRACE / LLAMA_SERVER_SLOTS_DEBUG / LLAMA_SERVER_SLOTS_N_DIFF env vars, sizes the shared llama_batch to max(n_batch, n_parallel), creates the RAM prompt cache, and initializes chat templates via common_chat_templates_init.
  tools/server/server-context.cpp#L1290-1460
• Chat completion trace: POST /v1/chat/completions -> routes.post_chat_completions -> handle_completions_impl -> tokenize/process_mtmd_prompt -> server_task(type=COMPLETION) per input -> rd.post_tasks -> server_queue::start_loop drains queue -> process_single_task -> get_available_slot -> launch_slot_with_task -> next update_slots() iteration batches it.
  tools/server/server-context.cpp#L4257-4342tools/server/server-queue.cpp#L278-316tools/server/server-context.cpp#L2382-2453
• Generation loop trace: update_slots() early-returns when all slots idle (else self-posts SERVER_TASK_TYPE_NEXT_RESPONSE) -> pre_decode() applies context-shift/checkpoints -> batch.render() -> for each n_batch chunk decode() -> queue_tasks.yield_to_queue wrapping llama_decode(ctx_tgt, batch_view) + llama_synchronize -> post_decode() samples and emits partial/final results.
  tools/server/server-context.cpp#L2807-2907tools/server/server-context.cpp#L3673-3686
• Streaming response trace: first rd.next() result decides headers; then res->set_next drives chunked output — format_oai_sse/format_anthropic_sse/format_oai_resp_sse per result, SSE keepalive ping ':' when sse_ping_interval elapses, 'data: [DONE]' terminator for OAI formats; errors mid-stream are formatted as SSE error events.
  tools/server/server-context.cpp#L4380-4523
• CLI startup trace: llama_cli -> common_init() -> common_params_parse(LLAMA_EXAMPLE_CLI) -> install SIGINT/SIGTERM handler -> cli_context(params) -> init(): either connect to --server-base or cli_server.start() spawns llama_server() on a std::thread with a random port, wait_ready() polls GET /health until 200, then client.wait_health and list_and_ask_models() (GET /v1/models) pick a model.
  tools/cli/cli.cpp#L33-64tools/cli/cli-server.h#L34-84tools/cli/cli-context.cpp#L94-183
• CLI turn trace: run() prints banner -> user_turn.read_input() -> slash-command dispatch (/regen pops last message, /clear resets, /image|/audio|/video stages base64, /read and /glob inline file text) -> push_user_message -> generate_completion POSTs SSE to /v1/chat/completions -> delta.reasoning_content/delta.content rendered via ui::assistant_turn, timings pulled from chunk.timings.
  tools/cli/cli-context.cpp#L413-560tools/cli/cli-context.cpp#L350-411
• Model load trace (persistence path): llama_model_load_from_file -> llama_model_load_from_file_impl validates exactly one of metadata/path/file, installs a default progress callback -> llama_model_load -> llama_model_loader ctor (gguf parse, split resolution, mmap) -> llama_model_create(ml, params) -> llama_prepare_model_devices -> model->load_hparams -> load_vocab -> load_stats -> print_info -> load_tensors (supports vocab_only early return and -2 cancellation via progress callback).
  src/llama.cpp#L316-444src/llama-model-loader.cpp#L80-105
• Router child spawn trace: server_models::load (server-models.cpp) builds child argv, sets subprocess_option_no_window|combined_stdout_stderr, calls inst.subproc->sproc.create(child_args, options, child_env); a single waiter thread watches all children, parses 'cmd_child_to_router:state:' lines from stdout and calls on_child_exit on EOF/exit; request_child_exit writes 'cmd_router_to_child:exit' to the child's stdin.
  tools/server/server-models.cpp#L45-80tools/server/server-models.cpp#L184-255tools/server/server-models.cpp#L1163-1280
• Shutdown trace: SIGINT -> signal_handler -> shutdown_handler -> (single-model) mcp_mgr.shutdown() + ctx_server.terminate() which unblocks start_loop -> clean_up(): server_stream_session_manager_stop, ctx_http.stop, ctx_server.terminate, llama_backend_free; a second Ctrl+C force-exits with exit(1).
  tools/server/server.cpp#L25-49tools/server/server.cpp#L448-558