openai/whisper
The repo is a single flat Python package, `whisper/`, plus a `tests/` pytest suite, GitHub Actions CI, and evaluation docs/notebooks. The CLI entrypoint `whisper.transcribe:cli` (also reachable via `python -m whisper`) parses ~30 argparse options, loads a checkpoint through `load_model` in `whisper/__init__.py`, and runs `transcribe()`. Checkpoints are runtime downloads from openaipublic.azureedge.net, verified by a SHA256 embedded in the URL and cached under ~/.cache/whisper. The model itself is a standard Transformer encoder/decoder defined in model.py; decoding.py implements the sampling engine (greedy/beam, logit filters, KV-cache hooks); audio.py, tokenizer.py, timing.py, utils.py, and triton_ops.py cover the audio pipeline, BPE vocab, DTW alignment, output writers, and optional CUDA kernels respectively.
- The console script `whisper` maps to `whisper.transcribe:cli` in pyproject `[project.scripts]`, and `whisper/__main__.py` calls the same `cli()` function, so `whisper` and `python -m whisper` are equivalent entrypoints.
- The package is organized as flat modules under whisper/: audio.py (mel pipeline), model.py (nn.Modules), decoding.py (sampling engine), tokenizer.py (tiktoken wrapper), timing.py (DTW alignment), transcribe.py (orchestration + CLI), utils.py (writers/helpers), normalizers/ (evaluation text normalization), triton_ops.py (CUDA kernels).
- whisper/__init__.py is the public API surface: it re-exports load_audio, log_mel_spectrogram, pad_or_trim, decode, detect_language, DecodingOptions, DecodingResult, Whisper, ModelDimensions, transcribe, and __version__, and holds the _MODELS download-URL table plus load_model().
- Model weights are downloaded at runtime from openaipublic.azureedge.net URLs in _MODELS; the expected SHA256 is the second-to-last URL path segment, checked both before reuse of a cached file and after download, defaulting to $XDG_CACHE_HOME/whisper or ~/.cache/whisper.
- Whisper is a conventional encoder-decoder Transformer: AudioEncoder applies two Conv1d layers (second stride 2), adds sinusoidal positional embeddings, then ResidualAttentionBlocks; TextDecoder uses learned positional embeddings, cross-attention blocks, a causal mask buffer, and tied output logits computed against token_embedding.weight.
- ffmpeg is a hard external dependency: load_audio() spawns `ffmpeg -nostdin -threads 0 -i <file> -f s16le -ac 1 -acodec pcm_s16le -ar 16000 -` and parses stdout as int16 scaled to float32.
- Declared dependencies are more-itertools, numba, numpy, tiktoken, torch, tqdm, and triton>=2 (Linux x86_64 only); Python >=3.8 is required.
- Binary assets ship inside the package: whisper/assets/gpt2.tiktoken and multilingual.tiktoken BPE vocab files, mel_filters.npz (80- and 128-band filterbanks), and normalizers/english.json (UK→US spelling map), all declared in MANIFEST.in.
- The Whisper class binds module-level functions as class attributes (`transcribe = transcribe_function`, `decode = decode_function`, `detect_language = detect_language_function`), so `model.transcribe(...)`, `model.decode(...)`, and `model.detect_language(...)` work without separate methods.
- Optional GPU acceleration lives in triton_ops.py (Triton JIT kernels for DTW and median filter); callers in timing.py wrap kernel launches in try/except and fall back to numba-JIT CPU implementations with a warning.
- Distribution is the PyPI package `openai-whisper` with dynamic version read from whisper/version.py (__version__ = "20250625"); python-publish.yml triggers on commits to main whose message matches `^Release <v>` and uploads via twine.
- Build/test: `pip install .[dev]` installs setuptools-built package plus pytest/black/flake8/isort/scipy; CI runs pre-commit then `pytest -k 'not test_transcribe or test_transcribe[tiny] or test_transcribe[tiny.en]' -m 'not requires_cuda'` on a 9-cell Python×PyTorch matrix.