# Testing signals — DietrichGebert/ponytail

> Canonical page: https://codevetter.com/unpack/ponytail/testing-signals

# Testing signals — DietrichGebert/ponytail

Scanned commit e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156 (2026-09-14). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Tests are plain node:test/assert suites — no framework, matching the project's own dogma. tests/*.test.js covers hook behavior end-to-end via spawnSync (every host env permutation), manifest/command/packaging drift guards, and grader self-checks; pi-extension/test and ponytail-mcp/test cover the adapters' pure functions. CI runs the suite plus two consistency scripts on every push/PR. Coverage is strong on the hooks protocol and drift guards; weaker on the Hermes Python plugin logic depth and on actual model behavior (benchmarks are manual, API-spend).
- Test framework is node:test + assert with no dependencies; `npm test` = `node --test tests/*.test.js && npm test --prefix pi-extension && npm test --prefix ponytail-mcp`. — [package.json#L37-38](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/package.json#L37-L38) [tests/commands.test.js#L8-9](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/commands.test.js#L8-L9) [pi-extension/package.json#L5-7](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/pi-extension/package.json#L5-L7)
- tests/hooks.test.js is an integration suite that spawnSyncs the real hook scripts under fabricated HOME/env dirs, asserting flag-file placement, per-host stdout JSON shapes, statusline nudge once-only behavior, CLAUDE_CONFIG_DIR override, and Copilot-vs-Codex state isolation. — [tests/hooks.test.js#L21-253](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/hooks.test.js#L21-L253)
- Subagent matcher behavior is fully covered: case-insensitive unanchored match, anchored rejection of superset names, missing agent_type fail-open, invalid-regex fail-open, and the no-matcher path's independence from stdin. — [tests/hooks.test.js#L292-358](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/hooks.test.js#L292-L358)
- Drift guards: tests/commands.test.js greps pi-extension/index.js for registerCommand calls and fails if any registered command lacks commands/<name>.toml or .opencode/command/<name>.md; tests/package.test.js guards scripts/uninstall.js staying in the npm files list. — [tests/commands.test.js#L17-39](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/commands.test.js#L17-L39) [tests/package.test.js](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/package.test.js)
- Per-host manifest tests exist for every adapter: gemini-extension.test.js, qoder-plugin.test.js, grok-plugin.test.js, hermes-plugin.test.js, copilot-plugin.test.js, opencode-plugin.test.js, openclaw-skills.test.js, plus hooks-windows.test.js and cursor-hooks.test.js for platform edges. — [tests/gemini-extension.test.js](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/gemini-extension.test.js) [tests/hermes-plugin.test.js](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/hermes-plugin.test.js) [tests/opencode-plugin.test.js](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/opencode-plugin.test.js) [tests/cursor-hooks.test.js](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/cursor-hooks.test.js) [tests/hooks-windows.test.js](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/hooks-windows.test.js)
- Two consistency scripts run in CI before tests: scripts/check-rule-copies.js byte-compares seven rule-file copies against AGENTS.md and asserts 9 rule invariants survive in SKILL.md+AGENTS.md; scripts/check-versions.js requires all 8 version-bearing manifests to share one pinned semver equal to the release tag. — [scripts/check-rule-copies.js#L19-76](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/scripts/check-rule-copies.js#L19-L76) [scripts/check-versions.js#L21-78](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/scripts/check-versions.js#L21-L78) [.github/workflows/test.yml#L29-36](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/.github/workflows/test.yml#L29-L36)
- Benchmark self-verification: `python benchmarks/agentic/run.py --selftest` validates every task scorer (good ref passes, bad ref caught on its declared axis), the plugin-dir resolver, and the tree-kill — before any API spend; tests/correctness.test.js and benchmarks/correctness.test.js cover scorer units. — [benchmarks/agentic/run.py#L183-244](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/benchmarks/agentic/run.py#L183-L244) [benchmarks/agentic/tasks.py#L15-21](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/benchmarks/agentic/tasks.py#L15-L21) [benchmarks/correctness.test.js](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/benchmarks/correctness.test.js)
- tests/behavior.test.js unit-tests the promptfoo behavior grader (benchmarks/behavior.js) by feeding known behavior-present/absent outputs through each probe checker, making the behavior.yaml eval trustworthy without an API key. — [tests/behavior.test.js#L1-50](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/behavior.test.js#L1-L50) [benchmarks/behavior.yaml](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/benchmarks/behavior.yaml)
- CI (.github/workflows/test.yml) runs on push to main, tags, and PRs: checkout → Node 22 + Python 3.12 + pandas → npm install for ponytail-mcp → check-rule-copies → check-versions → npm test. There is no coverage tool, lint step, or Windows/macOS CI matrix. — [.github/workflows/test.yml](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/.github/workflows/test.yml)
- The Hermes Python plugin (__init__.py) has a dedicated test (tests/hermes-plugin.test.js) but the deep logic — _filter_skill_body_for_mode parity with the JS version, gateway access denial — is exercised through JS-side assertions rather than a Python suite; there is no pytest configuration in the repo. (inferred) — [tests/hermes-plugin.test.js](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/hermes-plugin.test.js) [__init__.py#L70-122](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/__init__.py#L70-L122)
- Uncovered-by-default areas: live model behavior (benchmarks require ANTHROPIC_API_KEY/manual runs and are not in CI), the .ps1 statusline path on Windows (guarded only by hooks-windows.test.js shape checks), and whether hosts actually deliver injected context (docs/cursor-hooks.md notes that's a live-session check). (inferred) — [.env.example](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/.env.example) [benchmarks/README.md#L9-19](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/benchmarks/README.md#L9-L19) [tests/hooks-windows.test.js](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/tests/hooks-windows.test.js) [docs/cursor-hooks.md#L8-10](https://github.com/DietrichGebert/ponytail/blob/e3ba2aa6f1e6f0bc4d69eb09c9f0d0a93af56156/docs/cursor-hooks.md#L8-L10)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)