CodeVetter · Repo Unpack · Data flow
rust-lang/rust
Data enters as a source file (or string) plus CLI options, is lexed/parsed into an AST, expanded and lowered to HIR, then nearly everything else is demand-driven through the TyCtxt query system: type checking, trait solving, borrow checking, MIR construction/transforms, monomorphization, and finally codegen through a CodegenBackend trait to object files and a linked binary. Cross-crate state travels via rustc_metadata .rmeta blobs in rlibs/dylibs; incremental state is a serialized dep-graph+query cache; diagnostics flow out through rustc_errors' DiagCtxt. Bootstrap's data flow is stage0 binaries -> stage1 artifacts -> stage2 artifacts under build/.
- Input boundary: run_compiler builds an interface::Config from CLI matches — Input::File/Str, output dir/file, crate_cfg/crate_check_cfg, and an optional FileLoader; the default RealFileLoader reads from disk but the hook exists for e.g. rust-analyzer VFS.
- Text -> tokens -> AST: rustc_lexer produces tokens, rustc_parse (new_parser_from_source_str / passes::parse) produces ast::Crate; only the crate root is parsed initially, submodules are parsed during macro expansion.
- AST -> HIR: create_and_enter_global_ctxt forces tcx.resolver_for_lowering() which runs rustc_resolve + rustc_ast_lowering to produce resolutions and lowered HIR inside the GlobalCtxt; the 'tcx lifetime originates in this function's gcx_cell/arena setup.
- All mid/late compilation state is owned by the query system: queries.rs declares each query's key/value types and modifiers (caching, arena allocation, incremental), rustc_macros::rustc_queries generates TyCtxt accessors, and tcx.ensure_ok().analysis(()) drives typeck/borrowck/MIR construction.
- Provider wiring: each rustc_* crate exposes a provide() that fills fields of the generated Providers table; DEFAULT_QUERY_PROVIDERS chains ~25 such provide() calls, then codegen_backend.provide() and the Config::override_queries hook can overwrite individual providers before the GlobalCtxt is created.
- Codegen boundary: Linker::codegen_and_build_linker runs the backend's codegen queries producing object files/modules, then linker.link(sess, incr_comp_session, codegen_backend) runs outside the GlobalCtxt so the 'tcx arena can be freed earlier.
- Backend abstraction: rustc_codegen_ssa::traits::CodegenBackend defines the contract (name, init, target_config, provide, codegen methods); rustc_codegen_llvm::LlvmCodegenBackend is the default implementation bridging to LLVM via compiler/rustc_llvm FFI.
- Cross-crate metadata: rustc_metadata encodes crate metadata (EncodedMetadata) into .rmeta sections inside rlibs/dylibs and reads dependencies back through creader.rs/locator.rs/native_libs.rs, including the MetadataLoaderDyn interface the backend supplies.
- Incremental persistence: rustc_incremental::persist::{load,save,file_format,work_product,fs} manage an on-disk dep graph and query-result cache; create_and_enter_global_ctxt calls setup_dep_graph and rustc_incremental::load_query_result_cache before queries run.
- Dependency info output: passes::write_dep_info and passes::write_interface run inside the global-ctxt closure; Session exposes env_depinfo/file_depinfo that a driver's Config::track_state callback can populate for dep tracking.
- Diagnostics flow: everything reports through rustc_errors Diag/DiagCtxt built on DiagMessage (Str or inline Fluent); early errors before a Session exists go through EarlyDiagCtxt, and ICEs are captured by install_ice_hook writing a report to config.ice_file.
- Bootstrap data flow: bootstrap.py reads src/stage0 for pinned toolchain hashes and downloads from static.rust-lang.org/ci-artifacts; the Rust bootstrap reads bootstrap.toml (defaults in src/bootstrap/defaults, schema in bootstrap.example.toml) and produces staged artifacts under build/.
- Feature-gate state flows from rustc_feature's unstable.rs/accepted.rs/removed.rs tables into the session; tcx.report_unused_features() runs after codegen setup and -Zmetrics-dir dumps feature-usage metrics via dump_feature_usage_metrics.