CodeVetter · Repo Unpack evidence
Graphify-Labs/graphify
github.com/Graphify-Labs/graphify ↗ · 897 files · 11.1 MB
Read this in other languages 🇺🇸 English | 🇨🇳 简体中文 | 🇯🇵 日本語 | 🇰🇷 한국어 | 🇩🇪 Deutsch | 🇫🇷 Français | 🇪🇸 Español | 🇮🇳 हिन्दी | 🇧🇷 Português | 🇷🇺 Русский | 🇸🇦 العربية | 🇮🇷 فارسی | 🇮🇹 Italiano | 🇵🇱 Polski | 🇳🇱 Nederlands | 🇹🇷 Türkçe |
Overview
graphify (PyPI package `graphifyy`, CLI `graphify`) is a Python 3.10+ CLI plus an installable AI-assistant skill that turns a folder of code, docs, papers, images, and video/audio into a persistent, queryable knowledge graph under graphify-out/ (graph.json, graph.html, GRAPH_REPORT.md). Code is mapped locally and deterministically with ~30 tree-sitter grammars and per-language resolvers, while docs/media go through an optional semantic LLM pass (host-agent subagents or configured API backends). Every edge carries an EXTRACTED/INFERRED/AMBIGUOUS confidence tag, and the finished graph is queried through CLI subcommands (query/path/explain/affected/god-nodes) or an MCP server (stdio or HTTP). It targets users of 20+ coding assistants (Claude Code, Codex, Cursor, Gemini CLI, etc.) via generated per-platform skill bundles.
System map
A single-package Python CLI (`graphifyy`) structured as a staged pipeline: detect -> extract -> build -> cluster -> analyze -> report -> export, with each stage in its own module communicating through plain dicts and NetworkX graphs. The same library is driven two ways: headless via `graphify extract` subcommands in cli.py, or by an AI assistant following the packaged skill.md runbook that shells out to the library. Persistence is entirely file-based under graphify-out/ (graph.json, manifest.json, cache/, sidecars). Deployment surfaces are the PyPI wheel, per-platform skill files, and a Docker image that runs the MCP HTTP server.
- The canonical pipeline is detect() -> extract() -> build() -> cluster() -> analyze helpers -> report.generate() -> export.to_*(); each stage is a separate module exchanging plain dicts/NetworkX graphs with no shared state outside graphify-out/.
- Two console entry points ship in the wheel: `graphify` -> graphify.__main__:main and `graphify-mcp` -> graphify.serve:_main.
- graphify/__main__.py is a thin dispatcher: it re-exports the install subsystem from graphify/install.py (~90 names) and dispatch_command from graphify/cli.py, then main() wraps _run_cli() with broken-pipe handling.
- dispatch_command(cmd) in graphify/cli.py is the single command table for all non-install subcommands: provider, prs, hook, query, affected, god-nodes, save-result, reflect, path, explain, diagnose, add, watch, cluster-only/label, update, hook-check, hook-guard, check-update, tree, merge-driver, merge-graphs, clone, export, benchmark, global, extract, cache-check, merge-chunks, merge-semantic.
- All persistent state lives under the output directory `graphify-out`, overridable via the GRAPHIFY_OUT env var, centralized in graphify/paths.py (GRAPHIFY_OUT, out_path, default_graph_json) so security/path guards honor the same override.
- graph.json is a NetworkX node-link JSON file; serve.py _load_graph rehydrates it with json_graph.node_link_graph, forces directed=True for rendering, and attaches a work-memory overlay sidecar.
- AST extraction scales via concurrent.futures.ProcessPoolExecutor (_extract_parallel -> _extract_single_file worker, module-level for pickling), capped by cpu_count or GRAPHIFY_MAX_WORKERS, clamped to 61 on Windows, with sequential fallback on BrokenProcessPool/OSError.
- The MCP server is built once by serve.py::_build_server and served over stdio (serve()) or Streamable HTTP via starlette/uvicorn (serve_http), with optional X-API-Key/Bearer auth and a warning when binding 0.0.0.0 unauthenticated.
- The Dockerfile builds a python:3.12-slim image installing the [mcp] extra, runs as non-root uid 10001, and launches `python -m graphify.serve` with an HTTP transport default; graph.json is mounted at runtime, never baked in.
- Core dependencies are networkx, numpy, rapidfuzz, tree-sitter plus ~26 tree-sitter grammar packages; optional extras (pdf, office, video, mcp, neo4j, falkordb, sql, postgres, terraform, pascal, ocaml, commonlisp, robot, llm backends, all) gate niche extractors and integrations.
- Per-assistant skill bundles are generated artifacts: tools/skillgen/gen.py renders graphify/skill*.md, graphify/skills/<host>/references/, and graphify/always_on/*.md from tools/skillgen/fragments/, with CI drift guards (--check, --audit-coverage, --monolith-roundtrip).
- Publishing is trusted-publishing OIDC to PyPI on GitHub release (publish.yml), with a version==tag guard; release-graph.yml dogfoods the tool by building a graph of graphify itself and uploading graphify-self-graph.tar.gz as a release asset.
Feature catalog
User-facing surface = one CLI with ~30 subcommands, a generated `/graphify` skill for 20+ AI assistants, an MCP server exposing 10 graph tools, and a large set of export formats and corpus ingestors. Features span: corpus detection (code/docs/papers/images/video), AST extraction across ~40 language extensions, semantic (LLM) extraction with pluggable backends, community detection + report, multiple export sinks, git-hook auto-rebuild, cross-repo/global graphs, PR triage via `gh`, and work-memory reflection.
- `graphify extract <path>` is the headless full pipeline (detect -> AST -> semantic -> merge -> build -> cluster -> outputs) with flags --backend/--model/--mode deep/--out/--no-cluster/--code-only/--no-dedup/--postgres/--cargo/--max-workers/--token-budget/--exclude/--no-gitignore/--force/--allow-partial/--timing.
- Graph query commands: `query` (BFS/DFS with --budget/--context), `path` (shortest path), `explain`, `affected` (reverse traversal over DEFAULT_AFFECTED_RELATIONS incl. calls/imports/inherits/dynamic_import), `god-nodes`, and `diagnose multigraph`.
- Export formats via `graphify export <fmt>`: html, callflow-html, obsidian, wiki, svg, graphml, neo4j, falkordb; push_to_neo4j upserts via the neo4j driver with label/relation sanitization against Cypher injection.
- The MCP server registers 10 tools — query_graph, get_node, get_neighbors, get_community, god_nodes, graph_stats, shortest_path, list_prs, get_pr_impact, triage_prs — and injects an optional `project_path` into every schema for multi-project serving.
- `graphify install/uninstall` supports ~20 assistant platforms through _PLATFORM_CONFIG (claude, codex, opencode, kilo, aider, copilot, claw, droid, trae, trae-cn, hermes, kiro, pi, codebuddy, antigravity, windows, kimi, amp, ...), each mapping a generated skill-<platform>.md to a per-platform destination dir.
- `graphify hook install` writes post-commit and post-checkout git hooks that detect a working Python interpreter (pinned path, .graphify_python file, launcher shebang, uv tool envs, PATH fallback) and detached-launch `graphify.watch._rebuild_code` on changed files.
- `graphify update <path>` / `watch` / post-commit hooks all funnel into watch.py::_rebuild_code: AST-only incremental rebuild (no LLM), serialized by a per-repo flock with a .pending_changes queue drained by the lock holder.
- Semantic/LLM extraction supports backends claude, kimi, ollama, gemini, openai, deepseek, azure, bedrock, and a `claude-cli` path that shells to the claude binary; `graphify provider add/list/show/remove` registers custom OpenAI-compatible providers in a user-global JSON file.
- Non-code ingestors: mcp_ingest.py turns .mcp.json/claude_desktop_config.json into mcp_server/mcp_command/mcp_package/env_var nodes (env VALUES never read); manifest_ingest.py makes package manifests (pyproject, go.mod, pom.xml...) into depends_on edges; pg_introspect.py introspects live Postgres via psycopg; cargo_introspect.py handles Rust workspaces; transcribe.py does faster-whisper/yt-dlp video-to-text; google_workspace.py converts .gdoc/.gsheet shortcuts.
- Cross-repo features: `merge-graphs` union-merges graph.json files, `merge-driver` is a git merge driver for graph.json, and `graphify global add/remove/list` maintains a cross-repo graph at ~/.graphify/global-graph.json with per-repo tags.
- Work-memory features: `save-result` stores Q&A outcomes into graphify-out/memory/, and `graphify reflect` aggregates useful/dead_end/corrected signals with time-decay scoring into graphify-out/reflections/LESSONS.md (deterministic, no LLM).
- Auxiliary commands: `prs` (PR dashboard via gh CLI), `clone` (clone GitHub repo into ~/.graphify/repos), `add <url>` (fetch URL to ./raw with author/contributor tags), `benchmark` (corpus vs subgraph token comparison), `tree` (HTML tree view), `cache-check`, `cluster-only`, `label`, `check-update`, `merge-chunks`, `merge-semantic`, `diagnose`.
Data flow
Input boundary is detect(), which walks the corpus, classifies files into code/document/paper/image/video, and filters sensitive paths. Code files flow through extract(): suffix/filename/shebang dispatch to per-language extractors emitting {nodes, edges} dicts, then cross-file resolution passes (LanguageResolver registry) turn imports/member calls into INFERRED edges. Docs/papers/images flow through the semantic pass — either host-agent subagents writing chunk JSONs (skill.md path) or llm.extract_corpus_parallel packing token-budgeted chunks against a configured backend. Merged extractions are validated, deduplicated, built into a NetworkX graph, clustered, and exported to graphify-out/graph.json plus reports; incremental updates re-extract only changed files via manifest.json hashing.
- detect() classifies files via CODE_EXTENSIONS (~90 suffixes), DOC_EXTENSIONS, PAPER/IMAGE/VIDEO/OFFICE extension sets, a shebang interpreter check for extensionless executables, and filters _SENSITIVE_PATTERNS (.env, .pem, id_rsa, .npmrc, ...) and credential-store dirs (.ssh/.aws/.gnupg/.gcloud) before anything is read for extraction.
- Office/PDF inputs are screened as potential zip/XML bombs before parsing: _zip_within_caps enforces a 50 MiB raw cap, 512 MiB decompressed cap, and 200:1 compression-ratio limit with chunked streaming so the check itself never materializes the bomb.
- _get_extractor() dispatches a file to its extractor by exact filename first (.blade.php, MCP config, package manifests), then content sniffing for ambiguous suffixes (.h ObjC/C++ markers, .m ObjC-vs-MATLAB), then shebang for extensionless files (_SHEBANG_DISPATCH), then the _DISPATCH suffix table covering ~40 languages.
- Each extractor returns {nodes:[{id,label,source_file,source_location}], edges:[{source,target,relation,confidence}]}; validate.py::validate_extraction enforces this schema before build() consumes it, and confidence is EXTRACTED/INFERRED/AMBIGUOUS.
- extract() is two-phase: phase 1 checks the per-file AST cache (load_cached, keyed by content hash under cache/ast/v{pkg_version}-s{schema}/); phase 2 runs uncached files through ProcessPoolExecutor or sequentially, then runs registered LanguageResolver passes gated on which suffixes are present.
- The semantic pass wraps every file in a hash-stamped `<untrusted_source path=... sha256=...>` block, neutralizes known jailbreak tokens via _neutralise_injection_sentinels, packs chunks by tiktoken/char-estimate token budgets (_pack_chunks_by_tokens), and parses LLM output with _parse_llm_json's balanced-object salvage.
- The skill path's merge order is AST nodes first, then semantic nodes deduplicated by id — so a semantic node silently overwrites an AST node with the same id (richer labels win); build_from_json then mints `external` stub nodes for import-family edges whose targets are outside the graph.
- build.py collapses parallel edges to a single edge per (source,target) pair with a 'specific beats generic' precedence — references/uses/mentions lose to calls/imports/inherits — and drops cross-language-family phantom edges via _EDGE_LANG_FAMILY.
- cluster() partitions the graph (Leiden via graspologic or native backend, hub exclusion optional), analyze.py computes god_nodes/surprising_connections/suggest_questions/find_import_cycles, and report.generate() renders GRAPH_REPORT.md; export.to_json writes graph.json but refuses to overwrite a larger existing graph (shrink guard, bypassed by --force/GRAPHIFY_FORCE).
- Incremental updates are driven by graphify-out/manifest.json: detect_incremental diffs the corpus against stamped per-file hashes; _stamped_manifest_files only re-stamps semantic files that actually produced output so failed/partial extractions are re-queued next run; _stale_graph_sources prunes nodes whose source_file left the corpus.
- Read path: `graphify query` loads graph.json (size-capped by check_graph_file_size_cap, default 512 MiB, overridable via GRAPHIFY_MAX_GRAPH_BYTES), rebuilds an undirected NetworkX graph preserving per-edge _src/_tgt direction markers, runs _query_graph_text BFS/DFS, logs to querylog, and touches a query stamp.
- Post-commit hook flow: git hook shell block resolves a Python interpreter, detached-launches a python -c payload that sets a timeout watchdog, queues changed paths via .pending_changes, takes a non-blocking flock, and calls _rebuild_code; a concurrent commit's changes are merged in by the lock holder rather than dropped.
Behavior traces
Three traced flows: (1) `graphify install` copies the platform's generated skill bundle and registers hooks/always-on instruction blocks; (2) `graphify extract .` runs the full headless pipeline detect->extract->resolve->build->cluster->export with shrink guards; (3) `graphify query`/`graphify update` read and incrementally maintain graph.json. A fourth trace covers the git post-commit auto-rebuild. All flows converge on the same modules: detect/extract/build/cluster/report/export under graphify/.
- Startup trace (`graphify <cmd>`): __main__.main() -> _run_cli() reconfigures stdout/stderr to utf-8, checks every platform skill destination's .graphify_version stamp for staleness (skipped for install/uninstall/hook-check/hook-guard), prints usage if no args, then routes install commands to install.dispatch_install_cli and everything else to cli.dispatch_command.
- Install trace: dispatch_install_cli -> _canonical_platform/_PLATFORM_CONFIG lookup -> _platform_skill_destination -> _copy_skill_file copies graphify/skill-<host>.md (plus skills/<host>/references/ sidecar via _install_skill_references) into e.g. ~/.claude/skills/graphify/SKILL.md, then _write_version_stamp; hook platforms additionally write PreToolUse hook config (e.g. `graphify hook-guard` for Claude, `graphify hook-check` for Codex).
- Extract trace (headless): cli 'extract' parses flags -> persists corpus-shaping options (_write_build_cfg for excludes/gitignore) -> detect or detect_incremental -> collect_files -> extract() [cache check -> _extract_parallel/_extract_sequential -> run_language_resolvers] -> llm.extract_corpus_parallel for doc/image/paper chunks (unless --code-only) -> merge -> deduplicate_entities -> build_from_json -> cluster + score_all -> to_json (shrink-guarded) -> report.generate -> save_manifest.
- Skill-driven trace (`/graphify .`): skill.md instructs the host agent to resolve a Python interpreter into graphify-out/.graphify_python, run detect() writing .graphify_detect.json, run AST extraction to .graphify_ast.json, dispatch parallel general-purpose subagents writing .graphify_chunk_NN.json files (or call llm.extract_corpus_parallel when GEMINI_API_KEY/GOOGLE_API_KEY is set), merge to .graphify_semantic.json then .graphify_extract.json, then build_from_json -> cluster -> to_json -> GRAPH_REPORT.md, label communities, export html, save manifest, and clean temp files.
- Query trace: cli 'query' resolves --graph (default graphify-out/graph.json), enforces .json suffix + size cap, json.loads -> node_link_graph (undirected with per-edge _src/_tgt direction preserved) -> serve._query_graph_text(question, mode=bfs|dfs, depth, token_budget, context_filters) -> querylog.log_query -> _touch_query_stamp -> print.
- Update trace: cli 'update' reads GRAPHIFY_FORCE/--force, recovers the scan root from graphify-out/.graphify_root (utf-8-sig read), then watch._rebuild_code(watch_path, block_on_lock=True) -> _rebuild_lock flock -> detect() with persisted excludes -> re-extract changed code files only -> build_merge into existing graph -> to_json -> manifest update; doc/image changes still require the skill's --update path with an LLM.
- Post-commit hook trace: git hook -> _WORKTREE_GUARD skips linked worktrees -> _PYTHON_DETECT resolves interpreter (pinned exe -> .graphify_python -> launcher shebang -> uv tool dirs -> python3) -> _detached_launch spawns detached python -c rebuild -> _REBUILD_BODY_COMMIT sets SIGALRM/threading timeout (GRAPHIFY_REBUILD_TIMEOUT, kills multiprocessing children), validates .graphify_root stays in-repo, calls _rebuild_code(changed_paths), then best-effort reflect() to refresh LESSONS.md.
- MCP request trace: serve()/serve_http -> _build_server(graph_path) -> _GraphContextCache (pinned default + LRU of project graphs, GRAPHIFY_MAX_CONTEXTS default 8) -> tool call _select_graph(project_path) -> _load_ctx -> _load_graph (size cap + node_link_graph) -> handler (e.g. _tool_query_graph -> _query_graph_text) with ToolError mapped to isError results.
- Skill version-check trace: on any non-silent invocation, _run_cli iterates {_platform_skill_destination(name) for name in _PLATFORM_CONFIG} and calls _check_skill_version, which warns when the on-disk skill is older (suggests targeted `graphify install --platform X`) or NEWER than the package (warns that install would downgrade).
Testing signals
Testing is pytest-based with ~400 test files under tests/, one file per module plus many regression tests named after GitHub issues (test_issue_3405_*, etc.). Fixtures cover cross-language corpora (crate_a/crate_b Cargo workspaces, cpp/objc dirs, SQL samples). conftest.py sandboxes HOME and strips backend env vars so installers and detect_backend never touch the developer's real config. CI runs the suite on Python 3.10-3.14 plus a skillgen drift-guard job and non-blocking bandit/pip-audit scans.
- pytest is the test framework: pyproject sets testpaths=['tests'], norecursedirs excludes worked/ and corpus dirs, and the dev dependency group adds hypothesis, pytest-cov, ruff, pyright, bandit, pip-audit.
- conftest.py autouse fixtures give every test a throwaway HOME (protecting ~/.claude, ~/.agents, etc. from installer tests) and delete every env var detect_backend() consults so backend-detection tests are hermetic.
- ARCHITECTURE.md's module table is load-bearing documentation: tests/test_architecture_doc.py imports every symbol the doc names, so the architecture doc cannot drift from the code without a red test.
- The extractor migration is guarded by tests/test_extractors_registry.py — each ported language gets a module-import + facade-identity + registry-identity test before the move lands.
- tests/fixtures/ contains real mini-corpora: crate_a/crate_b Cargo workspaces for cross-crate call tests, cpp_paired/cpp_samedir dirs, objc_mixed, pascal_cross_file, robot_keywords.resource, sample_*.sql, and a committed extraction.json.
- CI test job runs `uv sync --all-extras --frozen` then `pytest tests/ -q --tb=short` on Python 3.10, 3.12, 3.13, 3.14, plus an install smoke test (`graphify --help`, `graphify install`).
- A separate skillgen-check CI job enforces that generated skill artifacts match their fragments: --check byte-diff, --audit-coverage per host, --schema-singleton file_type enum, --monolith-roundtrip and --always-on-roundtrip against the pinned v8 baseline commit.
- Security scanning exists but is advisory: bandit -r graphify -ll and pip-audit --strict both run with continue-on-error: true, so a new HIGH finding does not fail the build.
- Coverage breadth: dedicated test files exist for the CLI surface (test_cli_help, test_explain_cli, test_query_cli, test_path_cli, test_extract_cli), security (test_security, test_injection_sentinel_coverage), caching/incremental (test_cache, test_incremental, test_partial_cache), serve/MCP (test_serve, test_serve_http, test_query_mcp_direction), and ~40 per-language extraction tests (test_csharp_*, test_js_*, test_pascal_*, test_terraform, ...).
- Performance benchmarks live in-repo as tests/bench_extract.py and tests/bench_query_scoring.py, separate from the unit suite.
- Gap: the skill.md orchestration path (host-agent subagents writing chunk JSONs) is validated only indirectly via install-string and skillgen tests; no test exercises a real end-to-end /graphify run against a live assistant. The LLM client code is tested at the parsing/retry layer (test_llm_parser, test_llm_backends) but no live-API tests exist — consistent with the 'pure unit tests, no network calls' claim in ARCHITECTURE.md. (inferred)
- Regression-heavy style: dozens of tests are named for fixed issues (test_issue_3405_python_resolution, test_issue_3472_source_file_collision) and source comments cite issue numbers (#1392, #1666, #1941...) pervasively, meaning the test suite is the primary record of past failure modes.
Risk map
The main risks are (a) SSRF/SSRF-adjacent fetching — mitigated by a purpose-built guarded-connection stack; (b) prompt injection from attacker-controlled corpus files into the semantic LLM pass — mitigated but explicitly 'table stakes'; (c) the 8,300-line extract.py monolith mid-migration where the shared _extract_generic core and cross-file resolvers concentrate most defect density; (d) shell-out paths (git hooks, detached rebuilds, gh/git subprocesses, claude CLI) with heavy escaping/Windows edge-case handling; and (e) graph.json integrity guards (size cap, shrink guard, path validation) that protect but can also mask silent data loss if misread.
- All outbound fetching goes through security.py: validate_url allows only http/https, blocks private/reserved/loopback/link-local/CGN ranges and cloud metadata hosts, and _SSRFGuarded*Connection subclasses resolve+validate DNS once then connect to that exact IP to defeat DNS-rebind TOCTOU; redirects are re-validated by _NoFileRedirectHandler.
- Fetch size caps: safe_fetch aborts streaming reads at 50 MB (safe_fetch_text at 10 MB) and raises HTTPError on non-2xx; graph-file loads are capped at 512 MiB (GRAPHIFY_MAX_GRAPH_BYTES) to prevent memory-bomb json.loads.
- Prompt-injection surface: corpus files are wrapped in <untrusted_source sha256=...> blocks, _neutralise_injection_sentinels defangs chat-template tokens ([INST], <<SYS>>, forged closing tags), and sanitize_label/sanitize_metadata strip control chars + HTML-escape before export/MCP output — but SECURITY.md itself calls this 'table-stakes' and admits evasion is possible.
- detect.py silently skips credential-shaped files (.env, .pem/.key/.p12, id_rsa/id_ed25519, .npmrc/.pypirc/.git-credentials/.boto, secring) and refuses to descend into .ssh/.gnupg/.aws/.gcloud dirs — an attacker naming real source `id_rsa.py`-style or a user pointing graphify at a secrets dir still loses coverage silently.
- The HTTP MCP transport is opt-in and unauthenticated by default: serve_http warns loudly when binding 0.0.0.0 without --api-key/GRAPHIFY_API_KEY, and _ApiKeyMiddleware does a constant-time compare of X-API-Key/Bearer tokens — but a user can still expose the graph read-only to the network.
- extract.py is ~8,300 lines holding the shared _extract_generic core plus ~20 in-place cross-file resolution passes (member calls for swift/python/typescript/cpp/csharp/java/objc/kotlin/rust, qualified calls, interface dispatch); extractors/MIGRATION.md documents the ongoing one-language-per-PR extraction — this is the blast-radius hotspot where most issue-numbered regressions live.
- hooks.py embeds multi-stage shell+Python payloads in Python string constants with strict character constraints (no `"`, `$`, backtick, backslash inside the -c payload); a careless edit to _REBUILD_BODY_* or _LAUNCHER_TEMPLATE can silently break every installed hook — and the failure mode is a stale graph, not an error.
- Shell-outs: prs.py shells to `gh`/`git` (subprocess.run, no shell=True), cli._clone_repo runs git clone, llm._call_claude_cli shells to the claude binary, transcribe.py invokes yt-dlp — each is a supply-chain/PATH dependency that fails closed only if the binary is absent.
- install.py writes into user config dirs (~/.claude, ~/.codex, ~/.config/..., ~/.agents) and mutates assistant config files (settings.json hooks, AGENTS.md/CLAUDE.md sections) — a mis-install or stale uninstaller can corrupt the user's assistant setup; the conftest HOME sandbox exists precisely because this is dangerous in tests.
- scip_ingest.py is a self-declared skeleton ('NOT a full SCIP protobuf implementation ... Not wired to the CLI in this phase') — dead/legacy code carrying an ingestion surface with no caller or test coverage beyond import checks.
- Graph-integrity guards can mask failure modes: the to_json shrink guard (#479) refuses to overwrite a larger graph (good) but requires users to discover --force/GRAPHIFY_FORCE; the semantic 'unverified shrink' guard similarly stalls writes until --allow-partial; corrupt graph.json exits the process on CLI paths while MCP converts it to tool errors.
- Incremental correctness depends on manifest stamping discipline — a long trail of fixed bugs (#933, #1666, #1897, #1909, #1941, #2015, #2543) shows files could be silently marked done with no extraction output; _stamped_manifest_files is the single chokepoint an agent must not weaken.
Extension points
The codebase has clear, documented seams for extension: a suffix/filename/shebang dispatch table for new language extractors, a LanguageResolver registry for cross-file resolution passes, a BACKENDS dict + custom-provider JSON for LLM backends, a _PLATFORM_CONFIG table + skillgen fragments for new assistant platforms, and one-function-per-format exporters. ARCHITECTURE.md carries an explicit 5-step checklist for adding a language.
- New languages plug into extract.py's _DISPATCH suffix table (plus _SHEBANG_DISPATCH for extensionless, filename routers for MCP configs/package manifests, and content sniffs for ambiguous .h/.m); ARCHITECTURE.md lists the full checklist: extractor fn, _DISPATCH + collect_files registration, CODE_EXTENSIONS + watch _WATCHED_EXTENSIONS, pyproject grammar dep, fixture + tests.
- Cross-file/language-specific resolution passes register via resolver_registry.register(LanguageResolver(name, suffixes, resolve)) — e.g. terraform_modules, swift/python/ruby/typescript/csharp/java/rust member calls, kotlin import targets, csharp interface dispatch — and run_language_resolvers activates only resolvers whose suffixes appear in the corpus, logging-and-continuing on failure.
- extractors/ is the destination for new/migrated language extractors: MIGRATION.md prescribes verbatim moves, a mandatory facade re-export in extract.py, registry entry in extractors/__init__.py (LANGUAGE_EXTRACTORS seed), and a registry test — with strict 'never import graphify.extract from this package' directionality.
- LLM backends extend via the BACKENDS dict (base_url, default_model, env_key, pricing, temperature, vision) plus a call path (_call_openai_compat covers most); custom providers are added at runtime with `graphify provider add --base-url --default-model --env-key` persisted to a user-global JSON via _custom_providers_path.
- New assistant platforms extend _PLATFORM_CONFIG (skill_file, skill_dst, claude_md, skill_refs) and get generated artifacts by adding fragments under tools/skillgen/fragments/ + an entry in platforms.toml; install destinations and always-on blocks are resolved from packaged assets under graphify/skills/ and graphify/always_on/.
- Exporters are one-function-per-format in export.py (to_json, to_obsidian, to_canvas, to_graphml, to_svg, to_cypher) with html/callflow-html and neo4j/falkordb push in the exporters/ subpackage; a new format adds a function plus a `graphify export` subcommand arm.
- Optional-dependency gating is itself an extension seam: _EXTRA_FOR_EXTENSION maps suffixes to extras (.sql->sql, .tf->terraform, .ml->ocaml, .robot->robot...) so a grammar-backed language can ship as an extra without breaking the base install, and the #1745 warning names the missing extra to the user.
- The MCP tool list is extended by appending a types.Tool to the _tools list in _build_server and a handler in the dispatch dict — project_path is injected into every schema automatically, so new tools inherit multi-project support for free.
- New edge relations/relations-of-interest surface in affected traversal via DEFAULT_AFFECTED_RELATIONS and in build-time stub minting via _EXTERNAL_STUB_RELATIONS — both are small frozensets designed to be extended.
- Caching tiers are separately pluggable: the AST cache is versioned per package release + _AST_CACHE_SCHEMA bump, while the semantic cache is keyed by a fingerprint of the extraction prompt file — so changing the extraction-spec.md prompt invalidates semantic entries without touching AST entries.
Contributor handoff
Read ARCHITECTURE.md first — its module table is enforced by a test. Then skim __main__.py (dispatch + re-exports), cli.py::dispatch_command (command table), extract.py (_get_extractor/_DISPATCH + the resolver tail), and paths.py (output-dir constants, atomic writes, cross-platform path helpers). Run `uv sync --all-extras --frozen` then `pytest tests/ -q`. Conventions: issue-numbered comments, atomic writes via paths.write_*_atomic, never cache zero-node extractions, keep the __main__ re-export surface stable, and never edit generated skill*.md files — edit tools/skillgen/fragments and regen.
- First reads: ARCHITECTURE.md (pinned by tests/test_architecture_doc.py), graphify/__main__.py, graphify/cli.py dispatch_command, graphify/extract.py's _get_extractor/_DISPATCH and LanguageResolver registration block, graphify/paths.py, and graphify/security.py.
- Build/test: `uv sync --all-extras --frozen` (lock is committed; CI forbids re-resolving), `uv run --frozen pytest tests/ -q`; lint is a minimal ruff ruleset (E9,F63,F7,F82), pyright basic mode on graphify+tests; skill artifact drift check is `python -m tools.skillgen --check`.
- graphify/skill*.md, graphify/skills/*/references/*.md, and graphify/always_on/*.md are GENERATED — never hand-edit; change tools/skillgen/fragments/, run `python -m tools.skillgen`, and let --check/--audit-coverage/--monolith-roundtrip verify.
- Import direction rule: graphify/extract.py imports from graphify/extractors/, never the reverse; extractor ports must be verbatim byte-identical moves, one language per PR, with a facade re-export kept in extract.py.
- All file writes to graphify-out/ artifacts go through paths.write_text_atomic/write_json_atomic (temp file + os.replace + os_replace_with_fallback for Windows); never Path.write_text for graph.json/manifest/labels.
- Never cache a zero-node extraction result — extract.py deliberately skips save_cached when an extractable file returns no nodes so reruns self-heal; mirroring that, _stamped_manifest_files must stay the chokepoint deciding which files count as 'done'.
- extract(paths, root=...) : always pass root explicitly — without it node ids/source_file anchor to the common parent of the passed list, baking machine-specific path segments into portable graph.json artifacts.
- __main__.py's re-export list is a public API: install and cli symbols are re-imported there so `from graphify.__main__ import X` keeps working — do not remove names or move dispatch without preserving the re-exports.
- Stored/portable paths must be classified with paths.is_absolute_any_platform (POSIX+Windows), not Path.is_absolute — the corpus travels between OSes; but live-filesystem resolution keeps using Path.is_absolute.
- Edge/node conventions to preserve: confidence is exactly EXTRACTED|INFERRED|AMBIGUOUS; generic relations (references/uses/mentions) must lose edge-collapse to specific ones; import-family edges to undeclared targets mint `external` stub nodes; source_file is stored relative to the SCAN root.
- Trap: `graphify query` keeps the loaded graph undirected (BFS needs callers+callees) while path/explain force directed; edge direction lives in _src/_tgt per-link markers — do not 'simplify' either side without reading the #2309 comments.
- Hook/hook-guard payloads must stay shell-agnostic (pure `graphify` subcommand calls, no bash syntax) and the embedded python -c rebuild bodies must not contain `"`, `$`, backtick or backslash — constraints are documented inline and enforced by tests like test_hook_guard/test_hooks.
Languages
| Language | Files | Share of source |
|---|---|---|
| Python | 402 | 87% |
| Markdown | 362 | docs/config |
| C | 9 | 2% |
| JSON | 8 | docs/config |
| TypeScript | 8 | 2% |
| YAML | 8 | docs/config |
| C# | 7 | 2% |
| SQL | 7 | 2% |
Start reading here
Probable source entrypoint: pyproject.toml — inferred from the directory tree and manifest, not the upstream docs.
Source roots
graphify/ (223 files)
Repository structure
| Top-level path | Files |
|---|---|
tests/ | 405 |
graphify/ | 223 |
tools/ | 174 |
docs/ | 37 |
worked/ | 34 |
.github/ | 8 |
scripts/ | 1 |
Docs found: AGENTS.mdARCHITECTURE.mdREADME.mddocs/docker-mcp-sqlite.mddocs/how-it-works.mddocs/node-summaries-rfc.mddocs/translations/README.ar-SA.mddocs/translations/README.cs-CZ.mddocs/translations/README.da-DK.mddocs/translations/README.de-DE.md
Package
pyproject.toml — graphifyy v0.9.65, 56 declared dependencies.
#######mcpneo4jfalkordbpdfwatch
CI & config entrypoints
README.md— Repository readmeAGENTS.md— Agent instructions.github/workflows/publish.yml— GitHub Actions workflow.github/workflows/release-graph.yml— GitHub Actions workflow.github/workflows/ci.yml— GitHub Actions workflow
Structural health signals
Average score 6.7/10 across 96 analyzed source files; 40 hotspots surfaced.
graphify/dedup.py— 1020 lines, score 4.7 (Large file, Deep nesting)graphify/prs.py— 770 lines, score 5.2 (Large file, Deep nesting)graphify/llm.py— 1048 lines, score 5.4 (Large file, Deep nesting)
These are deterministic review leads — size, churn, and structural signals — not proof of a bug in this project.
Recent history
20a20d30d8e72026-09-20 — test(ts): de-flake the normalizer scaling test that reddened v8 CI7ca736cf94cd2026-09-20 — release: 0.9.65 — Pillow CVE floor, Go/Swift/Java/Verilog extractor fixes, JS block scoping, fail-closed eviction deadlock fix, vis-network crash + tooltip encoding, repo-link/docs86d8aadd3b082026-09-20 — fix(export): complete the vis-network crash fix and pin the tooltip XSS boundary (#3699, #3686 follow-ups)124c263488292026-09-20 — fix(export): prevent vis-network stack overflow crash on HTML render785ee5a426a82026-09-19 — Fix HTML entity encoding in graph tooltips