# Testing signals — Comfy-Org/ComfyUI

> Canonical page: https://codevetter.com/unpack/comfyui/testing-signals

# Testing signals — Comfy-Org/ComfyUI

Scanned commit b33e2b55cae074eca5aec96283cceac19aa249ba (2026-09-22). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Testing is pytest-based and split into two trees: tests-unit (fast unit tests, run in CI on Linux/Windows/macOS with CPU torch) and tests (integration: execution tests that spawn a real server subprocess and drive it over WebSocket, and inference tests that run full workflows and compare generated images against a baseline directory). Markers 'inference' and 'execution' allow deselection. Coverage of the node library itself is thin — most tests target the execution engine, validation, caching, and server endpoints rather than model math.
- pytest.ini sets testpaths to both tests and tests-unit, registers 'inference' and 'execution' markers, and adds -s (no output capture); pythonpath=. makes the repo root importable. — [pytest.ini#L1-9](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/pytest.ini#L1-L9)
- tests/execution/test_execution.py spins up the real server via subprocess.Popen([sys.executable, 'main.py', ...]), waits, connects a ComfyClient over websocket-client, submits prompt graphs built with GraphBuilder, and asserts on WS events and output images. — [tests/execution/test_execution.py#L1-30](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests/execution/test_execution.py#L1-L30) [tests/execution/test_execution.py#L505-580](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests/execution/test_execution.py#L505-L580)
- tests/execution also covers async nodes, jobs API, preview method override, progress isolation, and the public API (test_public_api.py), with a testing_nodes helper package of custom nodes for the tests. — [tests/execution](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests/execution)
- tests/inference/test_inference.py runs real diffusion workflows (ComfyGraph wrapper, KSampler imported from comfy.samplers) writing images to --output_dir; tests/compare performs quality regression by comparing directories, and conftest.py deliberately runs test_quality last. — [tests/inference/test_inference.py#L1-30](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests/inference/test_inference.py#L1-L30) [tests/conftest.py#L28-45](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests/conftest.py#L28-L45) [tests/README.md](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests/README.md)
- tests-unit covers the execution engine (test_execution_list, validate_node_input, loop validation, nested loops, cache providers, reentry), server/user endpoints (prompt_server_test, system_user_endpoint_test), folder_paths, security (security_test dir), assets, and comfy internals — all CPU-only. — [tests-unit/execution_test](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests-unit/execution_test) [tests-unit/prompt_server_test](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests-unit/prompt_server_test) [tests-unit/README.md](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests-unit/README.md)
- CI: .github/workflows/test-unit.yml runs `pytest tests-unit` on ubuntu-latest/windows-2022/macos-latest with Python 3.12 and CPU-only torch; it is marked continue-on-error, so unit failures do not block merges. — [.github/workflows/test-unit.yml](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/.github/workflows/test-unit.yml)
- Additional workflows: test-execution.yml and test-ci.yml run the execution/integration suites, test-launch.yml smoke-tests server launch (--quick-test-for-ci exits after startup), and ruff.yml lints with the rule set pinned in pyproject.toml. — [.github/workflows/test-execution.yml](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/.github/workflows/test-execution.yml) [.github/workflows/test-ci.yml](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/.github/workflows/test-ci.yml) [comfy/cli_args.py#L211](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/comfy/cli_args.py#L211) [pyproject.toml#L14-30](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/pyproject.toml#L14-L30)
- Test deps are separate: tests-unit/requirements.txt plus websocket-client/opencv/scikit-image for integration tests per tests/README.md; inference tests require downloaded models, so they are not runnable in plain CI. — [tests-unit/requirements.txt](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests-unit/requirements.txt) [tests/README.md](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests/README.md) [.github/workflows/test-unit.yml](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/.github/workflows/test-unit.yml)
- No coverage tooling, no mocked-HTTP layer for aiohttp routes, and no tests under comfy/ for individual model implementations — model correctness is only exercised indirectly through the heavyweight inference suite. (inferred) — [pytest.ini](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/pytest.ini) [tests](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests) [tests-unit](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/tests-unit)
- A --quick-test-for-ci flag in main.py exits right after the worker thread starts, which is what test-launch.yml uses to verify the server boots cleanly. — [main.py#L529-532](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/main.py#L529-L532) [comfy/cli_args.py#L211](https://github.com/Comfy-Org/ComfyUI/blob/b33e2b55cae074eca5aec96283cceac19aa249ba/comfy/cli_args.py#L211)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)