# Data flow — ggml-org/llama.cpp

> Canonical page: https://codevetter.com/unpack/llama-cpp/data-flow

# Data flow — ggml-org/llama.cpp

Scanned commit 7ab4ee7baad2d920464cbacfad4f4b07cf111fd2 (2026-09-22). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

The canonical pipeline is: CLI args -> common_params (arg.cpp) -> model file resolution/download (download.cpp, hf-cache.cpp) -> GGUF parse + mmap via llama_model_loader -> per-arch llama_model subclass (tensors mapped into ggml backend buffers) -> llama_context with KV memory + graph builder -> llama_decode produces logits -> sampler chain produces tokens -> detokenize to text. In the server, HTTP JSON is tokenized into server_task objects flowing through server_queue -> server_slot -> a shared llama_batch -> llama_decode -> results streamed back as SSE.
- Argument handling flows through common_params_parser_init in common/arg.cpp, which builds a table of common_arg entries each supporting env-var fallback (LLAMA_ARG_* / LLAMA_ARG_NO_*), per-example gating (LLAMA_EXAMPLE_SERVER/CLI/MTMD/TTS), and preset-only flags. — [common/arg.cpp#L60-141](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/arg.cpp#L60-L141) [common/arg.cpp#L3379-3505](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/arg.cpp#L3379-L3505)
- Model loading: common_init_from_params -> llama_model_load_from_file -> llama_model_load_from_file_impl -> llama_model_load constructs llama_model_loader (GGUF metadata + tensor index, mmap/mlock/direct-IO via params.load_mode), calls llama_model_create for arch dispatch, then load_hparams, load_vocab, load_stats, load_tensors in that order. — [common/common.cpp#L1329-1398](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/common.cpp#L1329-L1398) [src/llama.cpp#L316-378](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama.cpp#L316-L378) [src/llama.cpp#L465-470](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama.cpp#L465-L470)
- GGUF input is parsed by ggml/src/gguf.cpp and wrapped by llama_model_loader which resolves multi-file splits (names like name-00002-of-00004.gguf) via llama_get_list_splits and provides typed metadata access through the GGUFMeta template layer. — [src/llama-model-loader.cpp#L80-120](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-model-loader.cpp#L80-L120) [ggml/src/gguf.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/ggml/src/gguf.cpp)
- Device placement is decided in llama_prepare_model_devices: it partitions ggml_backend_dev_t devices into RPC servers (front of list), discrete GPUs, and iGPUs (only if no dGPU), dedupes by device_id, honors LLAMA_SPLIT_MODE_TENSOR by wrapping devices in a ggml_backend_meta_device, and trims to main_gpu for LLAMA_SPLIT_MODE_NONE. — [src/llama.cpp#L158-313](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama.cpp#L158-L313)
- Context creation in common_init_from_params loads LoRA adapters, initializes per-sequence samplers (common_sampler_init, optionally passed to the context for backend sampling via cparams.samplers), then calls llama_init_from_model which builds KV memory (llama_memory_i implementations: kv-cache, kv-cache-iswa, memory-hybrid, memory-recurrent) and the graph pipeline. — [common/common.cpp#L1343-1408](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/common.cpp#L1343-L1408) [src/llama-memory.h#L73-129](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-memory.h#L73-L129)
- Tokenization flows through llm_tokenizer implementations in src/llama-vocab.cpp (SPM, BPE with regex pre-tokenizers, UGM, etc.) selected by vocab type from GGUF metadata; unicode handling lives in src/unicode.cpp/unicode-data.cpp. — [src/llama-vocab.cpp#L73-286](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-vocab.cpp#L73-L286)
- Server request path: handle_completions_impl parses JSON, tokenizes via tokenize_input_prompts or process_mtmd_prompt (for multimodal), builds server_task objects with evaluated sampling params (server_schema::eval_llama_cmpl_schema), spawns child tasks when n_cmpl>1, and posts them to queue_tasks. — [tools/server/server-context.cpp#L4257-4342](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L4257-L4342) [tools/server/server-task.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-task.cpp)
- Slot scheduling: process_single_task handles COMPLETION/INFILL/EMBEDDING/RERANK by picking a slot via get_available_slot (deferring to queue_tasks.defer when none free or when child tasks lack slots) then launch_slot_with_task / launch_slots_with_parent_task. — [tools/server/server-context.cpp#L2375-2453](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L2375-L2453) [tools/server/server-queue.h#L53-77](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-queue.h#L53-L77)
- Decode loop: update_slots() runs pre_decode (context shift, checkpoint creation) -> batch.render() -> chunked decode() calls llama_decode(ctx_tgt, batch_view) inside queue_tasks.yield_to_queue (so metrics tasks can run mid-decode) -> post_decode per chunk; llama_decode itself is async so timings only settle after llama_synchronize. — [tools/server/server-context.cpp#L2793-2907](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L2793-L2907) [tools/server/server-context.cpp#L3647-3686](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L3647-L3686)
- Sampling is an ordered chain built by common_sampler_init: grammar/llg sampler first, then logit_bias, DRY, top-k, top-p, top-n-sigma, min-p, XTC, typical, temp_ext, infill, penalties, adaptive_p, and dist/temp at the end; chain order is data-driven from common_params_sampling. — [common/sampling.cpp#L187-404](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/common/sampling.cpp#L187-L404) [src/llama-sampler.cpp](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/src/llama-sampler.cpp)
- Results flow back through server_response queues: slots emit server_task_result_cmpl_partial/final which the HTTP layer formats as OAI SSE (format_oai_sse), Anthropic SSE (format_anthropic_sse), or OAI responses SSE, terminating with 'data: [DONE]'; non-stream requests collect via rd.wait_for_all. — [tools/server/server-context.cpp#L4348-4523](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-context.cpp#L4348-L4523) [tools/server/server-queue.h#L162-238](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/server/server-queue.h#L162-L238)
- CLI data flow: cli_context keeps an OpenAI-style messages array; user input (plus staged base64 media parts) is POSTed to /v1/chat/completions with stream=true and timings_per_token, and SSE deltas are rendered by ui::assistant_turn into reasoning/content streams. — [tools/cli/cli-context.cpp#L287-411](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/tools/cli/cli-context.cpp#L287-L411)
- Conversion pipeline data flow: convert_hf_to_gguf.py reads HF config/tokenizer/safetensors via a per-arch ModelBase subclass, maps tensor names through gguf MODEL_TENSORS tables in gguf-py/gguf/constants.py, and writes metadata+tensors to a .gguf file consumable by llama_model_loader. (inferred) — [convert_hf_to_gguf.py#L1-40](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/convert_hf_to_gguf.py#L1-L40) [gguf-py/gguf/constants.py](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/gguf-py/gguf/constants.py) [conversion/base.py](https://github.com/ggml-org/llama.cpp/blob/7ab4ee7baad2d920464cbacfad4f4b07cf111fd2/conversion/base.py)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)