# Testing signals — nextlevelbuilder/ui-ux-pro-max-skill

> Canonical page: https://codevetter.com/unpack/ui-ux-pro-max-skill/testing-signals

# Testing signals — nextlevelbuilder/ui-ux-pro-max-skill

Scanned commit dcc40ff5133ef78276117db0cc34e7b83cc8aeba (2026-09-21). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Testing is unusually heavy for a dataset repo: 13 stdlib unittest modules under src/ui-ux-pro-max/scripts/tests/ (mirrored into .claude/skills), a fixture-driven relevance evaluator with fingerprint-pinned thresholds, 5 Playwright e2e specs for the CLI (run as a Node test harness, not browser tests), node --test unit tests for the gallery, two bash smoke suites over all domains/stacks, and four validation scripts (CSV structure, semantic data contract, agent-guide claims, catalog summary). The aggregate `npm run verify:data` gate runs all of it and is wired into tests.yml, smoke-stacks.yml, release.yml, and prepublishOnly.
- Python tests live in src/ui-ux-pro-max/scripts/tests/ as 13 unittest modules (test_core, test_design_system_mode, test_data_contracts, test_style_taxonomy, test_catalog_refresh, test_relevance_evaluator, test_web_stack_freshness, etc.) deliberately stdlib-only so they add no dependencies; CI runs them via `python3 -m pytest .claude/skills -v`. — [src/ui-ux-pro-max/scripts/tests/test_core.py#L1-28](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/src/ui-ux-pro-max/scripts/tests/test_core.py#L1-L28) [.github/workflows/tests.yml#L52-53](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/.github/workflows/tests.yml#L52-L53)
- test_core.py covers tokenizer synonym/stopword behavior, BM25 edge cases, and cache invalidation by file signature (re-writes a temp CSV and asserts the index rebuilds). — [src/ui-ux-pro-max/scripts/tests/test_core.py#L31-80](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/src/ui-ux-pro-max/scripts/tests/test_core.py#L31-L80)
- The relevance harness uses fixtures (relevance-cases.json, relevance-thresholds.json, relevance-baseline.json) and graded judgments to compute routingAccuracy, precision@1/3, MRR@3, nDCG@3, negativeAbstention, typoRecoveryAt3, and designSystemCoherence, gated by check_thresholds with calibration vs held_out splits. — [scripts/evaluate-relevance.py#L96-138](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/scripts/evaluate-relevance.py#L96-L138) [scripts/relevance_metrics.py](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/scripts/relevance_metrics.py) [src/ui-ux-pro-max/scripts/tests/fixtures/relevance-cases.json](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/src/ui-ux-pro-max/scripts/tests/fixtures/relevance-cases.json)
- Hard-negative cases tagged 'hard-negative' are run against every domain and every stack to measure false-positive abstention (globalNegativeApplicability), so retrieval regressions that over-match are caught by the gate. — [scripts/evaluate-relevance.py#L144-163](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/scripts/evaluate-relevance.py#L144-L163)
- validate_data.py is a 1095-line semantic contract: it enforces exactly-192-row parity across products/colors/ui-reasoning, WCAG contrast floors per palette pair, style status/parent/redirect invariants, decision-rule validity against the closed grammar, icon import regexes, and per-stack official-doc host allowlists. — [src/ui-ux-pro-max/scripts/validate_data.py#L75-145](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/src/ui-ux-pro-max/scripts/validate_data.py#L75-L145) [src/ui-ux-pro-max/scripts/validate_data.py#L231-414](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/src/ui-ux-pro-max/scripts/validate_data.py#L231-L414)
- validate-csv.py checks structural CSV hygiene (blank/duplicate headers, field-count mismatches, blank rows) across every data CSV; validate-agent-guide.py recomputes the advertised counts (styles, palettes, fonts, stacks) and asserts the SKILL.md/template descriptions and CLI examples stay accurate. — [scripts/validate-csv.py#L26-89](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/scripts/validate-csv.py#L26-L89) [scripts/validate-agent-guide.py#L21-76](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/scripts/validate-agent-guide.py#L21-L76)
- smoke-domains.sh dynamically enumerates CSV_CONFIG domains (expected count 12 via EXPECTED_DOMAIN_COUNT) and asserts a focused probe returns >=1 result per domain, printing diagnostics on failure; smoke-stacks.sh does the same for stacks. — [scripts/smoke-domains.sh#L9-86](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/scripts/smoke-domains.sh#L9-L86) [scripts/smoke-stacks.sh](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/scripts/smoke-stacks.sh)
- CLI e2e specs (cli/tests/e2e/) use @playwright/test as a plain Node harness: dry-run.spec.ts asserts planPlatformInstallActions lists writes but touches nothing, template-script-paths.spec.ts and agent-query-contract.spec.ts pin the rendered skill content contract; playwright.config.ts targets chromium with CI retries. — [cli/tests/e2e/dry-run.spec.ts#L10-40](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/cli/tests/e2e/dry-run.spec.ts#L10-L40) [cli/playwright.config.ts](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/cli/playwright.config.ts) [cli/tests/e2e/template-script-paths.spec.ts](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/cli/tests/e2e/template-script-paths.spec.ts)
- Gallery tests run via `tsc -p tsconfig.test.json && node --test` over compiled JS (lib/parseStyles.test.js, components/metadata-badges.test.js) — an unusual compile-then-test pipeline rather than jest/vitest. — [gallery/package.json#L8](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/gallery/package.json#L8) [gallery/lib/parseStyles.test.ts](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/gallery/lib/parseStyles.test.ts) [gallery/components/metadata-badges.test.tsx](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/gallery/components/metadata-badges.test.tsx)
- check-asset-sync.yml embeds two grep-based contract suites: skill-relative path enforcement (no ~/.claude/skills/, project-rooted, or CLAUDE_PLUGIN_ROOT paths outside the one allowed SKILL.md) and a bundled-skill allowlist scan over every shipped markdown file. — [.github/workflows/check-asset-sync.yml#L41-156](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/.github/workflows/check-asset-sync.yml#L41-L156)
- Coverage gaps: no tests exercise the CLI's actual write path (generatePlatformFiles is only tested via the read-only planners), uninstall.ts has no spec, and format_ascii_box/markdown output is only implicitly covered through persistence tests. (inferred) — [cli/tests/e2e/dry-run.spec.ts](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/cli/tests/e2e/dry-run.spec.ts) [cli/src/commands/uninstall.ts](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/cli/src/commands/uninstall.ts) [cli/src/utils/template.ts#L286-323](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/cli/src/utils/template.ts#L286-L323)
- release.yml runs the full gate (verify:data, bun build, Playwright, gallery build) before semantic-release publishes to npm with NPM_TOKEN and creates the GitHub release — so untested code cannot ship. — [.github/workflows/release.yml#L49-90](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill/blob/dcc40ff5133ef78276117db0cc34e7b83cc8aeba/.github/workflows/release.yml#L49-L90)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)