CodeVetter · Repo Unpack · Testing signals
deepseek-ai/DeepSeek-V3
There is effectively no test infrastructure: no test directory, no test framework in requirements.txt, and no CI workflow that builds or runs code — the sole GitHub Actions job is a stale-issue bot. Verification relies on a __main__ smoke block in model.py, a hardcoded warmup generation at startup, and liberal use of runtime asserts for shape/divisibility/contiguity preconditions. The README's benchmark tables are external evaluation results, not runnable tests.
- No test files or directories exist in the repo; requirements.txt contains only runtime deps (torch, triton, transformers, safetensors) with no pytest, and the only workflow is stale.yml, which runs actions/stale and no code.
- model.py's __main__ block is the only executable self-check: it builds a default Transformer on CUDA with a random [2,128] token batch and prints logits size — it asserts nothing and requires a GPU to run.
- generate.py performs a warmup call generate(model, [tokenizer.encode('DeepSeek')], 2, -1, 1.) before weights are loaded, which smoke-tests allocation and the forward path but is not a correctness assertion (eos_id=-1 is a placeholder).
- Defensive asserts are used as the main correctness mechanism: vocab/out/in features and expert counts must divide by world_size, tensors must be contiguous, and scale/weight dims must be 2-D.
- fp8_gemm_kernel is wrapped in triton.autotune over 36 configs (BLOCK_SIZE_M x BLOCK_SIZE_N x num_stages) keyed on N,K — a performance-regression mechanism, not a correctness test.
- The .gitignore is a stock Python template (references pytest_cache, coverage, tox, Django/Flask artifacts) that does not reflect any tooling actually present — e.g., no safetensors or checkpoint patterns are ignored.
- Benchmark tables in README.md (MMLU, HumanEval, AIME, etc.) are published results from external evaluation harnesses; nothing in the repo reproduces them.
- Issue hygiene is semi-automated: stale.yml marks issues stale after 30 days and closes after 14, exempting 'pinned' and 'security' labels; PRs are exempt (days-before-pr-stale: -1).
- Untested critical surface includes the entire convert.py shard/rename pipeline and the Gate routing math — a silent mismatch there yields wrong weights or wrong routing with no failure signal. (inferred)