CodeVetter · Repo Unpack · Testing signals
AUTOMATIC1111/stable-diffusion-webui
Testing is API-level only: pytest + requests against a live server started with --test-server on CPU using test/test_files/empty.pt as the checkpoint. test/ holds 6 test modules covering txt2img, img2img, extras, face restorers, torch utils, and endpoint smoke tests. CI (run_tests.yaml) launches the real server on ubuntu-latest with coverage, runs pytest with pytest-base-url, then kills it via the /sdapi/v1/server-stop route. Lint CI runs ruff (pinned 0.3.3) and eslint. There are no unit tests of the sampling pipeline, no UI tests, and coverage of modules/ internals is incidental.
- Test framework is pytest 7.3 + pytest-base-url + pytest-cov (requirements-test.txt); pyproject.toml [tool.pytest.ini_options] sets base_url=http://127.0.0.1:7860; tests use plain requests calls, no fixtures beyond conftest base64 image encoders and an 'initialize' fixture that imports webui.
- Tests are end-to-end HTTP tests: test_txt2img.py POSTs a minimal 64x64 3-step 'Euler a' request to /sdapi/v1/txt2img; test_img2img.py covers img2img/inpaint modes with base64 fixtures; test_utils.py smoke-tests GET endpoints (samplers, sd-models, embeddings, ...) and a read-write /sdapi/v1/options roundtrip.
- CI runs the actual app: run_tests.yaml does 'python launch.py --skip-torch-cuda-test --exit' to prepare env, then starts 'launch.py --test-server --use-cpu all --api-server-stop' under coverage, waits on port 7860, runs pytest --verify-base-url test, and stops the server via curl -XPOST /sdapi/v1/server-stop.
- --test-server mode mutates sys.argv to inject --api, --ckpt test/test_files/empty.pt, --skip-torch-cuda-test, --disable-nan-check and blanks COMMANDLINE_ARGS, letting CI run generation-shaped calls without a real model or GPU.
- conftest.py sets IGNORE_CMD_ARGS_ERRORS=1 so pytest's own argv doesn't break shared_cmd_options' argparse parse (which otherwise calls parser.parse_args() at import time).
- Lint coverage: on_pull_request.yaml runs ruff==0.3.3 (config in pyproject.toml, target py39, selects B/C/I/W, excludes extensions/) and eslint via 'npm run lint' (eslint ^8.40.0, scripts lint/fix in package.json).
- Coverage artifacts are produced (coverage combine + html uploaded as artifacts), but nothing enforces a coverage floor; the core diff math — sd_samplers_kdiffusion, sd_samplers_cfg_denoiser, prompt_parser, sd_hijack — has no direct unit tests and is only exercised indirectly through the 6 API test files. (inferred)
- test/test_files contains tiny fixtures (empty.pt fake checkpoint, img2img_basic.png, mask_basic.png, two-faces.jpg); test_outputs/ is an output scratch dir — there is no golden-image regression comparison of generated outputs.