# Data flow — Graphify-Labs/graphify

> Canonical page: https://codevetter.com/unpack/graphify/data-flow

# Data flow — Graphify-Labs/graphify

Scanned commit 20a20d30d8e7eef77675651f0199d87f913bd3e7 (2026-09-20). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Input boundary is detect(), which walks the corpus, classifies files into code/document/paper/image/video, and filters sensitive paths. Code files flow through extract(): suffix/filename/shebang dispatch to per-language extractors emitting {nodes, edges} dicts, then cross-file resolution passes (LanguageResolver registry) turn imports/member calls into INFERRED edges. Docs/papers/images flow through the semantic pass — either host-agent subagents writing chunk JSONs (skill.md path) or llm.extract_corpus_parallel packing token-budgeted chunks against a configured backend. Merged extractions are validated, deduplicated, built into a NetworkX graph, clustered, and exported to graphify-out/graph.json plus reports; incremental updates re-extract only changed files via manifest.json hashing.
- detect() classifies files via CODE_EXTENSIONS (~90 suffixes), DOC_EXTENSIONS, PAPER/IMAGE/VIDEO/OFFICE extension sets, a shebang interpreter check for extensionless executables, and filters _SENSITIVE_PATTERNS (.env, .pem, id_rsa, .npmrc, ...) and credential-store dirs (.ssh/.aws/.gnupg/.gcloud) before anything is read for extraction. — [graphify/detect.py#L25-49](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/detect.py#L25-L49) [graphify/detect.py#L110-147](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/detect.py#L110-L147)
- Office/PDF inputs are screened as potential zip/XML bombs before parsing: _zip_within_caps enforces a 50 MiB raw cap, 512 MiB decompressed cap, and 200:1 compression-ratio limit with chunked streaming so the check itself never materializes the bomb. — [graphify/detect.py#L55-108](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/detect.py#L55-L108)
- _get_extractor() dispatches a file to its extractor by exact filename first (.blade.php, MCP config, package manifests), then content sniffing for ambiguous suffixes (.h ObjC/C++ markers, .m ObjC-vs-MATLAB), then shebang for extensionless files (_SHEBANG_DISPATCH), then the _DISPATCH suffix table covering ~40 languages. — [graphify/extract.py#L6502-6545](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/extract.py#L6502-L6545) [graphify/extract.py#L6270-6373](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/extract.py#L6270-L6373) [graphify/extract.py#L6411-6426](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/extract.py#L6411-L6426)
- Each extractor returns {nodes:[{id,label,source_file,source_location}], edges:[{source,target,relation,confidence}]}; validate.py::validate_extraction enforces this schema before build() consumes it, and confidence is EXTRACTED/INFERRED/AMBIGUOUS. — [ARCHITECTURE.md#L50-73](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/ARCHITECTURE.md#L50-L73) [graphify/validate.py#L10](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/validate.py#L10)
- extract() is two-phase: phase 1 checks the per-file AST cache (load_cached, keyed by content hash under cache/ast/v{pkg_version}-s{schema}/); phase 2 runs uncached files through ProcessPoolExecutor or sequentially, then runs registered LanguageResolver passes gated on which suffixes are present. — [graphify/extract.py#L6787-6918](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/extract.py#L6787-L6918) [graphify/resolver_registry.py#L59-85](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/resolver_registry.py#L59-L85) [graphify/cache.py#L22-40](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/cache.py#L22-L40)
- The semantic pass wraps every file in a hash-stamped `<untrusted_source path=... sha256=...>` block, neutralizes known jailbreak tokens via _neutralise_injection_sentinels, packs chunks by tiktoken/char-estimate token budgets (_pack_chunks_by_tokens), and parses LLM output with _parse_llm_json's balanced-object salvage. — [graphify/llm.py#L28-38](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/llm.py#L28-L38) [graphify/llm.py#L572-660](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/llm.py#L572-L660) [graphify/llm.py#L2143-2203](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/llm.py#L2143-L2203) [graphify/llm.py#L1037-1199](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/llm.py#L1037-L1199)
- The skill path's merge order is AST nodes first, then semantic nodes deduplicated by id — so a semantic node silently overwrites an AST node with the same id (richer labels win); build_from_json then mints `external` stub nodes for import-family edges whose targets are outside the graph. — [graphify/build.py#L1-21](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/build.py#L1-L21) [graphify/build.py#L75-137](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/build.py#L75-L137) [graphify/skill.md#L361-391](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/skill.md#L361-L391)
- build.py collapses parallel edges to a single edge per (source,target) pair with a 'specific beats generic' precedence — references/uses/mentions lose to calls/imports/inherits — and drops cross-language-family phantom edges via _EDGE_LANG_FAMILY. — [graphify/build.py#L56-77](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/build.py#L56-L77) [graphify/build.py#L139-150](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/build.py#L139-L150)
- cluster() partitions the graph (Leiden via graspologic or native backend, hub exclusion optional), analyze.py computes god_nodes/surprising_connections/suggest_questions/find_import_cycles, and report.generate() renders GRAPH_REPORT.md; export.to_json writes graph.json but refuses to overwrite a larger existing graph (shrink guard, bypassed by --force/GRAPHIFY_FORCE). — [graphify/cluster.py#L232-373](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/cluster.py#L232-L373) [graphify/analyze.py#L109-180](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/analyze.py#L109-L180) [graphify/export.py#L229-271](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/export.py#L229-L271) [graphify/skill.md#L413-437](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/skill.md#L413-L437)
- Incremental updates are driven by graphify-out/manifest.json: detect_incremental diffs the corpus against stamped per-file hashes; _stamped_manifest_files only re-stamps semantic files that actually produced output so failed/partial extractions are re-queued next run; _stale_graph_sources prunes nodes whose source_file left the corpus. — [graphify/cli.py#L88-168](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/cli.py#L88-L168) [graphify/cli.py#L221-250](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/cli.py#L221-L250) [graphify/manifest.py#L1-4](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/manifest.py#L1-L4)
- Read path: `graphify query` loads graph.json (size-capped by check_graph_file_size_cap, default 512 MiB, overridable via GRAPHIFY_MAX_GRAPH_BYTES), rebuilds an undirected NetworkX graph preserving per-edge _src/_tgt direction markers, runs _query_graph_text BFS/DFS, logs to querylog, and touches a query stamp. — [graphify/cli.py#L1244-1322](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/cli.py#L1244-L1322) [graphify/security.py#L32-66](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/security.py#L32-L66) [graphify/security.py#L357-383](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/security.py#L357-L383)
- Post-commit hook flow: git hook shell block resolves a Python interpreter, detached-launches a python -c payload that sets a timeout watchdog, queues changed paths via .pending_changes, takes a non-blocking flock, and calls _rebuild_code; a concurrent commit's changes are merged in by the lock holder rather than dropped. — [graphify/hooks.py#L144-352](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/hooks.py#L144-L352) [graphify/watch.py#L166-224](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/watch.py#L166-L224) [graphify/watch.py#L1355-1441](https://github.com/Graphify-Labs/graphify/blob/20a20d30d8e7eef77675651f0199d87f913bd3e7/graphify/watch.py#L1355-L1441)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)