# Testing signals — deepseek-ai/DeepSeek-V3

> Canonical page: https://codevetter.com/unpack/deepseek-v3/testing-signals

# Testing signals — deepseek-ai/DeepSeek-V3

Scanned commit 9b4e9788e4a3a731f7567338ed15d3ec549ce03b (2025-08-28). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

There is effectively no test infrastructure: no test directory, no test framework in requirements.txt, and no CI workflow that builds or runs code — the sole GitHub Actions job is a stale-issue bot. Verification relies on a __main__ smoke block in model.py, a hardcoded warmup generation at startup, and liberal use of runtime asserts for shape/divisibility/contiguity preconditions. The README's benchmark tables are external evaluation results, not runnable tests.
- No test files or directories exist in the repo; requirements.txt contains only runtime deps (torch, triton, transformers, safetensors) with no pytest, and the only workflow is stale.yml, which runs actions/stale and no code. — [inference/requirements.txt](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/requirements.txt) [.github/workflows/stale.yml](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/.github/workflows/stale.yml)
- model.py's __main__ block is the only executable self-check: it builds a default Transformer on CUDA with a random [2,128] token batch and prints logits size — it asserts nothing and requires a GPU to run. — [inference/model.py#L801-808](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L801-L808)
- generate.py performs a warmup call generate(model, [tokenizer.encode('DeepSeek')], 2, -1, 1.) before weights are loaded, which smoke-tests allocation and the forward path but is not a correctness assertion (eos_id=-1 is a placeholder). — [inference/generate.py#L117-118](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/generate.py#L117-L118)
- Defensive asserts are used as the main correctness mechanism: vocab/out/in features and expert counts must divide by world_size, tensors must be contiguous, and scale/weight dims must be 2-D. — [inference/model.py#L101](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L101) [inference/model.py#L219](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L219) [inference/model.py#L248](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L248) [inference/model.py#L658](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L658) [inference/kernel.py#L51-52](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/kernel.py#L51-L52) [inference/kernel.py#L104-105](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/kernel.py#L104-L105)
- fp8_gemm_kernel is wrapped in triton.autotune over 36 configs (BLOCK_SIZE_M x BLOCK_SIZE_N x num_stages) keyed on N,K — a performance-regression mechanism, not a correctness test. — [inference/kernel.py#L113-118](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/kernel.py#L113-L118)
- The .gitignore is a stock Python template (references pytest_cache, coverage, tox, Django/Flask artifacts) that does not reflect any tooling actually present — e.g., no safetensors or checkpoint patterns are ignored. — [.gitignore](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/.gitignore)
- Benchmark tables in README.md (MMLU, HumanEval, AIME, etc.) are published results from external evaluation harnesses; nothing in the repo reproduces them. — [README.md#L105-219](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/README.md#L105-L219)
- Issue hygiene is semi-automated: stale.yml marks issues stale after 30 days and closes after 14, exempting 'pinned' and 'security' labels; PRs are exempt (days-before-pr-stale: -1). — [.github/workflows/stale.yml#L14-29](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/.github/workflows/stale.yml#L14-L29)
- Untested critical surface includes the entire convert.py shard/rename pipeline and the Gate routing math — a silent mismatch there yields wrong weights or wrong routing with no failure signal. (inferred) — [inference/convert.py](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/convert.py) [inference/model.py#L566-598](https://github.com/deepseek-ai/DeepSeek-V3/blob/9b4e9788e4a3a731f7567338ed15d3ec549ce03b/inference/model.py#L566-L598)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)