CodeVetter · Repo Unpack · Feature catalog
pytorch/pytorch
User-facing surface is huge: eager tensor ops bound from _VariableFunctions, the nn.Module/optim training stack, tape-based autograd, torch.compile (Dynamo+Inductor), torch.export and legacy TorchScript for graph capture, torch.distributed with pluggable collectives backends plus the torchrun elastic launcher, checkpointing via torch.save/load, data loading and multiprocessing helpers, quantization/sparse/fft/linalg/profiler subpackages, and C++ extension authoring via cpp_extension. New features typically surface either as torch.* functions (codegen-bound aten ops) or as subpackage imports in torch/__init__.py.
- Eager tensor ops are exposed by copying every member of _C._VariableFunctions into the torch namespace at import (PRIVATE_OPS like unique_dim are skipped; __all__ is extended for non-underscore names).
- torch.nn.Module is the model base class providing parameters(), state_dict()/load_state_dict(), and the __call__ -> forward dispatch used by all neural network code.
- torch.autograd.backward() and torch.autograd.grad() drive the tape-based engine: backward validates inputs, handles __torch_function__ overrides, builds grad tensors via _make_grads, then calls _engine_run_backward which invokes Variable._execution_engine.run_backward in C++.
- torch.compile(model, fullgraph=, backend=, mode=, options=, dynamic=, recompile_limit=, isolate_recompiles=, dynamic_shapes=) is the JIT entry point: it wraps the backend string (default inductor) in _TorchCompileInductorWrapper/_TorchCompileWrapper and returns torch._dynamo.optimize(...)(model).
- compile modes are "default", "reduce-overhead" (CUDA graphs), "max-autotune" and "max-autotune-no-cudagraphs"; options keys include epilogue_fusion, max_autotune, fallback_random, shape_padding, triton.cudagraphs, trace.enabled and guard_filter_fn.
- torch.export.export()/save()/load() provide AoT graph capture producing ExportedPrograms; torch.compile is explicitly a no-op inside an export region.
- TorchScript lives on via torch.jit (torch/jit/_script.py) and mobile .ptl artifacts; SECURITY.md treats them as executable code requiring trust.
- torch.distributed.init_process_group supports gloo/nccl/mpi/ucc/xccl and third-party backends, auto-selecting nccl for cuda, gloo for cpu, xccl for xpu when backend is omitted; collectives run over a Store established by rendezvous or init_method URL.
- torch.save writes a zip container (records: data.pkl, .format_version, .storage_alignment, byteorder, data/<key> storages); torch.load reads it with weights_only=True by default since 2.6, supporting map_location, mmap and a safe-globals allowlist.
- Accelerator and utility subpackages imported at the end of __init__: cpu, cuda, mps, mtia, xpu, amp, autocast, distributions, fft, linalg, nested, sparse, special, profiler, hub, futures, multiprocessing, optim, nn, ao (quantization), quasirandom, masked.
- Higher-order and functional transforms (vmap, cond, sym_* operators) are part of the public API listed in torch.__all__.
- Custom C++/CUDA extensions are built JIT via torch.utils.cpp_extension.load()/load_inline() which generate ninja build files and compile into loadable modules.
- Determinism, dtype and RNG controls are torch-level functions calling into _C setters: use_deterministic_algorithms, set_float32_matmul_precision, set_default_dtype/_tensor_type, manual_seed, get/set_rng_state.