ggml-org/llama.cpp

The canonical pipeline is: CLI args -> common_params (arg.cpp) -> model file resolution/download (download.cpp, hf-cache.cpp) -> GGUF parse + mmap via llama_model_loader -> per-arch llama_model subclass (tensors mapped into ggml backend buffers) -> llama_context with KV memory + graph builder -> llama_decode produces logits -> sampler chain produces tokens -> detokenize to text. In the server, HTTP JSON is tokenized into server_task objects flowing through server_queue -> server_slot -> a shared llama_batch -> llama_decode -> results streamed back as SSE.

• Argument handling flows through common_params_parser_init in common/arg.cpp, which builds a table of common_arg entries each supporting env-var fallback (LLAMA_ARG_* / LLAMA_ARG_NO_*), per-example gating (LLAMA_EXAMPLE_SERVER/CLI/MTMD/TTS), and preset-only flags.
  common/arg.cpp#L60-141common/arg.cpp#L3379-3505
• Model loading: common_init_from_params -> llama_model_load_from_file -> llama_model_load_from_file_impl -> llama_model_load constructs llama_model_loader (GGUF metadata + tensor index, mmap/mlock/direct-IO via params.load_mode), calls llama_model_create for arch dispatch, then load_hparams, load_vocab, load_stats, load_tensors in that order.
  common/common.cpp#L1329-1398src/llama.cpp#L316-378src/llama.cpp#L465-470
• GGUF input is parsed by ggml/src/gguf.cpp and wrapped by llama_model_loader which resolves multi-file splits (names like name-00002-of-00004.gguf) via llama_get_list_splits and provides typed metadata access through the GGUFMeta template layer.
  src/llama-model-loader.cpp#L80-120ggml/src/gguf.cpp
• Device placement is decided in llama_prepare_model_devices: it partitions ggml_backend_dev_t devices into RPC servers (front of list), discrete GPUs, and iGPUs (only if no dGPU), dedupes by device_id, honors LLAMA_SPLIT_MODE_TENSOR by wrapping devices in a ggml_backend_meta_device, and trims to main_gpu for LLAMA_SPLIT_MODE_NONE.
  src/llama.cpp#L158-313
• Context creation in common_init_from_params loads LoRA adapters, initializes per-sequence samplers (common_sampler_init, optionally passed to the context for backend sampling via cparams.samplers), then calls llama_init_from_model which builds KV memory (llama_memory_i implementations: kv-cache, kv-cache-iswa, memory-hybrid, memory-recurrent) and the graph pipeline.
  common/common.cpp#L1343-1408src/llama-memory.h#L73-129
• Tokenization flows through llm_tokenizer implementations in src/llama-vocab.cpp (SPM, BPE with regex pre-tokenizers, UGM, etc.) selected by vocab type from GGUF metadata; unicode handling lives in src/unicode.cpp/unicode-data.cpp.
  src/llama-vocab.cpp#L73-286
• Server request path: handle_completions_impl parses JSON, tokenizes via tokenize_input_prompts or process_mtmd_prompt (for multimodal), builds server_task objects with evaluated sampling params (server_schema::eval_llama_cmpl_schema), spawns child tasks when n_cmpl>1, and posts them to queue_tasks.
  tools/server/server-context.cpp#L4257-4342tools/server/server-task.cpp
• Slot scheduling: process_single_task handles COMPLETION/INFILL/EMBEDDING/RERANK by picking a slot via get_available_slot (deferring to queue_tasks.defer when none free or when child tasks lack slots) then launch_slot_with_task / launch_slots_with_parent_task.
  tools/server/server-context.cpp#L2375-2453tools/server/server-queue.h#L53-77
• Decode loop: update_slots() runs pre_decode (context shift, checkpoint creation) -> batch.render() -> chunked decode() calls llama_decode(ctx_tgt, batch_view) inside queue_tasks.yield_to_queue (so metrics tasks can run mid-decode) -> post_decode per chunk; llama_decode itself is async so timings only settle after llama_synchronize.
  tools/server/server-context.cpp#L2793-2907tools/server/server-context.cpp#L3647-3686
• Sampling is an ordered chain built by common_sampler_init: grammar/llg sampler first, then logit_bias, DRY, top-k, top-p, top-n-sigma, min-p, XTC, typical, temp_ext, infill, penalties, adaptive_p, and dist/temp at the end; chain order is data-driven from common_params_sampling.
  common/sampling.cpp#L187-404src/llama-sampler.cpp
• Results flow back through server_response queues: slots emit server_task_result_cmpl_partial/final which the HTTP layer formats as OAI SSE (format_oai_sse), Anthropic SSE (format_anthropic_sse), or OAI responses SSE, terminating with 'data: [DONE]'; non-stream requests collect via rd.wait_for_all.
  tools/server/server-context.cpp#L4348-4523tools/server/server-queue.h#L162-238
• CLI data flow: cli_context keeps an OpenAI-style messages array; user input (plus staged base64 media parts) is POSTed to /v1/chat/completions with stream=true and timings_per_token, and SSE deltas are rendered by ui::assistant_turn into reasoning/content streams.
  tools/cli/cli-context.cpp#L287-411
• Conversion pipeline data flow: convert_hf_to_gguf.py reads HF config/tokenizer/safetensors via a per-arch ModelBase subclass, maps tensor names through gguf MODEL_TENSORS tables in gguf-py/gguf/constants.py, and writes metadata+tensors to a .gguf file consumable by llama_model_loader. (inferred)
  convert_hf_to_gguf.py#L1-40gguf-py/gguf/constants.pyconversion/base.py