# Behavior traces — deepseek-ai/DeepSeek-V3

> Canonical page: https://codevetter.com/unpack/deepseek-v3/behavior-traces

# Behavior traces — deepseek-ai/DeepSeek-V3

Scanned commit 9b4e9788e4a3a731f7567338ed15d3ec549ce03b (2025-08-28). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Three flows were traced end-to-end. Startup: torchrun env -> process group -> config -> model construction -> warmup -> weight load. Interactive turn: input -> broadcast -> chat template -> generate loop -> decode -> history append. Per-decode-step forward: embed -> freqs_cis slice -> per-Block (attn_norm -> MLA -> residual; ffn_norm -> MLP/MoE -> residual) -> final norm -> head -> all-gather -> sample. Offline conversion runs glob -> safe_open -> rename -> shard/expert-partition -> save_file -> tokenizer copy.
- Startup trace: torchrun sets WORLD_SIZE/RANK/LOCAL_RANK -> main() reads them via os.getenv -> dist.init_process_group('nccl') if world_size>1 -> non-rank-0 rebinds print to a no-op -> torch.cuda.set_device(local_rank) -> set_default_dtype(bfloat16) -> ModelArgs loaded from --config JSON. — [inference/generate.py#L100-114](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/generate.py#L100-L114)
- Startup continues: Transformer(args) constructor sets globals world_size/rank from torch.distributed and Linear.dtype (float8_e4m3fn if args.dtype=='fp8' else bfloat16), then builds embed/layers/norm/head and precomputes freqs_cis. — [inference/model.py#L750-770](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L750-L770)
- Startup finishes with a warmup: tokenizer loaded via AutoTokenizer, generate() called on the tokenized string 'DeepSeek' for 2 tokens, then safetensors load_model applies the rank's shard into the already-allocated module tree. — [inference/generate.py#L115-119](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/generate.py#L115-L119)
- Interactive turn trace: input('>>> ') on rank 0 -> dist.broadcast_object_list propagates to other ranks -> '/exit' breaks, '/clear' empties messages -> messages.append(user) -> apply_chat_template -> generate() -> tokenizer.decode(skip_special_tokens=True) -> print -> messages.append(assistant). — [inference/generate.py#L121-144](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/generate.py#L121-L144)
- Decode-step trace inside generate(): for each cur_pos, model.forward(tokens[:, prev_pos:cur_pos], prev_pos) -> Transformer.forward embeds tokens, slices freqs_cis[start_pos:start_pos+seqlen], builds causal mask if seqlen>1, runs every Block, applies final norm on the last position only (h[:, -1]), projects to logits, all_gathers across ranks. — [inference/generate.py#L60-69](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/generate.py#L60-L69) [inference/model.py#L772-798](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L772-L798)
- MLA 'absorb' trace: wq_a -> q_norm -> wq_b -> view to heads -> split q_nope/q_pe -> apply_rotary_emb(q_pe) -> wkv_a -> split kv/k_pe -> apply_rotary_emb(k_pe) -> kv_norm(kv) written to kv_cache, k_pe to pe_cache -> scores = einsum(q_nope,kv_cache)+einsum(q_pe,pe_cache) -> softmax fp32 -> einsum with kv_cache then wkv_b value slice -> wo. — [inference/model.py#L461-497](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L461-L497)
- Gate trace: linear(x, gate.weight) -> softmax-or-sigmoid scores -> add bias if present -> if n_groups>1, compute group scores (amax without bias, topk(2) sum with bias), keep top topk_groups groups via masked_fill(-inf) -> topk(n_activated_experts) indices -> gather weights from pre-bias scores -> sigmoid weights normalized -> *= route_scale. — [inference/model.py#L576-598](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L576-L598)
- MoE trace: gate(x) -> bincount per-expert counts -> for each local expert index with count>0, torch.where(indices==i) selects tokens and y[idx] += expert(x[idx]) * weight -> shared_experts MLP on all tokens -> dist.all_reduce(y) -> return (y+z).view(shape). — [inference/model.py#L679-693](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L679-L693)
- Conversion trace: glob(hf_ckpt_path/*.safetensors) -> safe_open each file -> per tensor: skip 'model.layers.61', strip 'model.' prefix, apply name replacements, look up (new_key, shard_dim) in mapping -> for each rank i: filter experts to that rank's contiguous range or narrow the tensor along shard_dim -> save_file per rank -> copy *token* files. — [inference/convert.py#L50-85](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/convert.py#L50-L85)
- fp8->bf16 trace: load index.json weight_map -> sort *.safetensors -> per file: load to cuda, drop *_scale_inv entries, weight_dequant each 1-byte tensor with its scale_inv (warning + passthrough on KeyError) -> save_file -> prune scale_inv keys from weight_map -> write new index.json. — [inference/fp8_cast_bf16.py#L63-103](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/fp8_cast_bf16.py#L63-L103)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)