CodeVetter · Repo Unpack · Extension points
deepseek-ai/DeepSeek-V3
This is a reference implementation, not a plugin architecture — there are no registries or hook interfaces. Extension happens through four narrow seams: the ModelArgs/configs JSON surface for new model sizes, the convert.py `mapping` table for new HF weight names, the module-level Literal switches (gemm_impl, attn_impl, score_func, scale_fmt) for implementation variants, and the fp8_gemm_configs autotune list for kernel tuning. New chat commands are string-matched literals in the interactive loop.
- ModelArgs is the single configuration surface: adding support for a new checkpoint size means adding a JSON file under inference/configs/ whose keys match ModelArgs field names — unknown keys would raise TypeError in ModelArgs(**json.load(f)).
- convert.py's `mapping` dict is the extension table for weight renames: to support a new HF parameter name, add an entry mapping the HF key segment to (demo_name, shard_axis_or_None); unmapped keys hit 'assert key in mapping'.
- attn_impl is a Literal['naive','absorb'] global switch; a third attention backend would slot into MLA.__init__'s cache-buffer branch and MLA.forward's score/output branches.
- gemm_impl is a Literal['bf16','fp8'] switch consulted in the linear() helper; a new GEMM path plugs in there and applies globally to every Linear subclass.
- Gate.score_func is a Literal['softmax','sigmoid'] switch with per-branch logic in forward (scoring and weight normalization); adding a scorer means touching both places.
- scale_fmt is threaded from ModelArgs -> Linear.scale_fmt -> linear() -> act_quant -> act_quant_kernel constexpr; 'ue8m0' is the only implemented format (power-of-two scales via ceil(log2)), so new scale formats extend kernel.py's branch.
- fp8_gemm_configs is an explicit list of triton.Config tile parameters; new shapes/tunings are added by editing this list, and @triton.autotune keys on N,K.
- Chat slash commands are hardcoded string comparisons in the interactive loop ('/exit', '/clear'); a new command is another elif before messages.append.
- Block selects dense-vs-MoE FFN purely by layer_id < args.n_dense_layers; architectures with interleaved or partially-MoE layouts require changing this branch, not just config.
- Experts are constructed per-rank with None placeholders in the ModuleList for non-local indices — extending to expert-parallel offload or remote experts would hook at this construction and the dispatch loop.
- There is no plugin/registry system; all behavior changes are source edits. The repo's own recommendation for anything beyond demo use is to switch frameworks (SGLang/vLLM/LMDeploy/TRT-LLM/LightLLM). (inferred)