CodeVetter · Repo Unpack evidence
AUTOMATIC1111/stable-diffusion-webui
github.com/AUTOMATIC1111/stable-diffusion-webui ↗ · 303 files · 2.4 MB
Stable Diffusion web UI A web interface for Stable Diffusion, implemented using Gradio library.
Overview
AUTOMATIC1111 stable-diffusion-webui is a self-hosted Gradio web UI and FastAPI REST server for running Stable Diffusion (SD1.x/SD2/SDXL/SD3) image generation locally. Its two entry paths — webui.webui() for the Gradio UI and webui.api_only() for a bare FastAPI app — both funnel into the same modules/processing.py pipeline driven by a StableDiffusionProcessing dataclass. Its most distinctive trait is a deep extension system: Python 'scripts', per-file extensions under extensions/ and extensions-builtin/, a ~20-category callback registry in modules/script_callbacks.py, and prompt-embedded 'extra networks' (<lora:...>, <hypernet:...>) that mutate the model at generation time.
System map
A single-process Python application with a thin shell launcher (webui.sh/webui.bat) that runs launch.py, which delegates to modules/launch_utils.py to install torch/pip requirements, git-clone pinned dependency repos into repositories/, then imports webui.py. webui.py either builds a Gradio Blocks UI (modules/ui.py create_ui) plus optional REST routes (modules/api/api.py, mounted only with --api/--nowebui), or runs FastAPI-only mode on port 7861. All generation state lives in module-level globals in modules/shared.py (shared.sd_model, shared.opts, shared.state). Checkpoints are loaded by modules/sd_models.py through modules/safe.py's monkeypatched torch.load; settings persist to config.json via modules/options.py; user data roots are computed in modules/paths_internal.py.
- launch.py is the real entrypoint: main() optionally dumps sysinfo, calls launch_utils.prepare_environment() (pip installs, repo clones, extension installers) unless --skip-prepare-environment, optionally configure_for_tests() for --test-server, then start() which imports webui and calls webui() or api_only().
- webui.py's webui() runs an outer while-loop that calls initialize.initialize() once, then repeatedly: ui_tempdir.cleanup_tmpdr(), script_callbacks.before_ui_callback(), shared.demo = ui.create_ui(), shared.demo.queue(64), shared.demo.launch(...), then blocks on shared.state.wait_for_server_command(timeout=5) for 'stop'/'restart' to support in-process UI restart.
- The REST API is an APIRouter-free design: modules/api/api.py's Api class registers ~40 /sdapi/v1/* routes directly on the FastAPI app inside webui.create_api(app), gated behind --api (alongside UI) or --nowebui (api_only, port default 7861).
- modules/shared.py is a global state bag: it holds cmd_opts, opts (Options), sd_model, state (shared_state.State), demo (gr.Blocks), prompt_styles, total_tqdm, mem_mon, and re-exports paths/constants; nearly every module reads/writes these globals.
- modules/paths_internal.py computes all storage roots at import time: script_path (repo root), data_path (--data-dir, default repo root), models_path (--models-dir or data_path/models), extensions_dir (data_path/extensions), extensions_builtin_dir (script_path/extensions-builtin), and reads COMMANDLINE_ARGS env var into sys.argv.
- launch_utils.prepare_environment() installs torch (TORCH_COMMAND/TORCH_INDEX_URL env overridable, default cu121 wheels torch==2.1.2), verifies CUDA unless --skip-torch-cuda-test, installs clip/open_clip/xformers, and git-clones five pinned dependency repos (assets, stablediffusion, generative-models, k-diffusion, BLIP) into repositories/ at pinned commit hashes.
- initialize.initialize() (modules/initialize.py) is the heavyweight boot: fix_torch_version/tls/sigint utils, sd_models.setup_model(), codeformer and gfpgan model dirs, then initialize_rest() which sets samplers, lists extensions, restores config state, lists models/localizations, calls scripts.load_scripts(), loads upscalers/VAEs/embeddings, registers extra networks, and spawns a background Thread to load the SD model via shared.sd_model.
- The Gradio UI is built in modules/ui.py create_ui() which constructs gr.Blocks for txt2img, img2img (with sub-tabs: img2img/Sketch/Inpaint/Inpaint sketch/Inpaint upload/Batch), Extras, PNG Info, Train, Checkpoint Merger, then appends script-provided tabs via script_callbacks.ui_tabs_callback() plus built-in Settings and Extensions tabs, sorted by opts.ui_tab_order.
- Generation work is serialized through a single modules/call_queue.py queue_lock (a FIFOLock): wrap_gradio_gpu_call wraps every GPU-bound gradio handler and the API shares the same lock via Api(app, queue_lock), so only one generation job runs at a time.
- Settings are owned by modules/options.py Options class (data_labels OptionInfo registry defined in modules/shared_options.py) persisted as JSON to shared.config_filename (--ui-settings-file, default data_path/config.json); Options.load() contains migration code for legacy keys (sd_vae_as_default, quicksettings, ui_reorder).
- modules/sd_models.py manages the checkpoint registry (checkpoints_list of CheckpointInfo), list_models() scans ckpt dirs, load_model() builds the model via instantiate_from_config + LoadStateDictOnMeta, applies fp16 conversion, and calls sd_hijack.model_hijack.hijack(sd_model); torch.load is globally monkeypatched by modules/safe.py to run a pickle-safety scan (check_pt) unless --disable-safe-unpickle.
- Deployment is user-run local install: webui.sh/webui.bat create/activate a venv (venv_dir configurable in webui-user.sh) and exec launch.py; there is no packaging, docker, or service config in the repo — CI runs it as a plain process on port 7860.
Feature catalog
User-facing features split across Gradio tabs (txt2img, img2img with six sub-modes, Extras upscaling/face-fix, PNG Info, Train for embeddings/hypernetworks, Checkpoint Merger, Settings, Extensions manager) and a ~40-route /sdapi/v1 REST API. Core generation supports txt2img/img2img/inpainting, hires-fix (second-pass upscale), ~30 samplers across k-diffusion/timesteps/LCM families, prompt scheduling and extra-network syntax, selectable scripts (xyz_grid, prompt_matrix, loopback, outpainting, sd_upscale, custom_code), and training of textual-inversion embeddings and hypernetworks.
- txt2img: UI calls modules/txt2img.py txt2img() → txt2img_create_processing() builds StableDiffusionProcessingTxt2Img with prompt/negative/styles/batch/cfg/hires-fix params (enable_hr, hr_scale, hr_upscaler, hr_second_pass_steps, hr_checkpoint_name) and per-request override_settings.
- img2img + inpainting: modules/img2img.py builds StableDiffusionProcessingImg2Img (init_images, mask, denoising_strength, inpainting fields); the Gradio tab exposes six modes (img2img, Sketch, Inpaint, Inpaint sketch, Inpaint upload, Batch).
- REST API exposes POST /sdapi/v1/txt2img, /img2img, /extra-single-image, /extra-batch-images, /png-info, /interrogate, /interrupt, /skip, plus GETs for options, samplers, schedulers, upscalers, sd-models, sd-vae, hypernetworks, face-restorers, realesrgan-models, prompt-styles, embeddings, memory, scripts, script-info, extensions; POSTs for options, refresh-*, create/train embedding+hypernetwork, unload/reload-checkpoint.
- API auth is optional HTTP Basic via --api-auth 'user:pass,...' checked with secrets.compare_digest; the API also accepts image payloads as base64 or http(s) URLs, the latter gated by opts.api_enable_requests and SSRF-checked by verify_url() + opts.api_forbid_local_requests.
- Sampler catalog is assembled in modules/sd_samplers.py from three families: sd_samplers_kdiffusion.samplers_data_k_diffusion, sd_samplers_timesteps.samplers_data_timesteps, sd_samplers_lcm.samplers_data_lcm; schedulers (Automatic/Karras/Exponential/etc.) live in modules/sd_schedulers.py and can be encoded in the sampler name suffix ('Euler a Karras').
- Prompt language supports attention weights ((word), (word:1.2)), prompt editing/scheduling ([a:b:step], [a|b] alternation) parsed by modules/prompt_parser.py get_learned_conditioning_prompt_schedules/parse_prompt_attention, and extra-network tags <name:args> extracted by extra_networks.parse_prompt.
- Built-in 'scripts' in scripts/ are user-selectable generation programs: xyz_grid.py (X/Y/Z parameter plots), prompt_matrix.py, loopback.py, sd_upscale.py, outpainting_mk_2.py, poor_mans_outpainting.py, img2imgalt.py, prompts_from_file.py, custom_code.py (arbitrary Python, requires --allow-code), plus postprocessing_* wrappers.
- Extras tab does single/batch image postprocessing (upscale via ESRGAN/RealESRGAN/SwinIR/LDSR/ScuNET/DAT upscalers, GFPGAN/CodeFormer face restoration) via modules/postprocessing.py and modules/ui_postprocessing.py, exposed to API as /sdapi/v1/extra-single-image and /extra-batch-images.
- Train tab + API can create and train textual-inversion embeddings (textual_inversion.create_embedding/train_embedding) and hypernetworks (create_hypernetwork/train_hypernetwork), with preview images routed through txt2img params.
- PNG Info tab + /sdapi/v1/png-info parse generation parameters ('infotext') embedded in PNG text chunks or JPEG/WEBP EXIF UserComment; modules/infotext_utils.py handles parse_generation_parameters and send-to-txt2img/img2img paste.
- Extra Networks UI (modules/ui_extra_networks.py) renders browsable card pages for checkpoints, textual inversion, hypernetworks, and Lora (registered by extensions-builtin/Lora) with per-file .json user metadata (extra_networks.get_user_metadata) served as in-app HTML pages via add_pages_to_demo.
- Checkpoint Merger tab (modules/ui_checkpoint_merger.py) merges two/three checkpoints into a new .safetensors file; Settings tab (modules/ui_settings.py) edits the OptionInfo registry with quicksettings bar, search, and per-setting apply/restart semantics.
Data flow
Input boundary is either Gradio component values or pydantic API models (modules/api/models.py). Both converge on a StableDiffusionProcessing dataclass ('p') that is the single mutable carrier through the pipeline: scripts mutate it, prompt parsing expands it, sampling consumes it, and a Processed object carries results back out. Persistence flows: settings → config.json, generated images + infotext → outpath_* dirs, last prompt → data_path/params.txt, checkpoint sha256 cache → cache.json, per-model metadata → <name>.json sidecars.
- UI path: Gradio button handlers call modules/txt2img.py or modules/img2img.py which build StableDiffusionProcessingTxt2Img/Img2Img with shared.sd_model and outpath_samples from opts.outdir_* settings, attach p.scripts=scripts_txt2img/img2img runner and p.script_args (the flattened list of all script UI component values).
- API path: text2imgapi/img2imgapi convert the pydantic request via apply_infotext() (merges parsed infotext fields the caller didn't set), init_script_args() (merges script_name + alwayson_scripts args into the flat script_args list at args_from/args_to indexes), then call process_images directly under the shared queue_lock.
- process_images() snapshots overridden opts, applies p.override_settings via opts.set(k,v,is_api=True) — special-casing sd_model_checkpoint (reload_model_weights) and sd_vae (reload_vae_weights) — applies token merging, fixes invalid sampler/scheduler names, then runs process_images_inner() under profiling.Profiler and restores opts in a finally block.
- process_images_inner() order: fix seeds → fill fields from opts → setup_prompts() → load embeddings → p.scripts.process → p.init() once → per-iteration loop: slice prompts/seeds → before_process_batch → parse_extra_network_prompts → extra_networks.activate → process_batch → setup_conds → p.sample() → post_sample → decode_latent_batch → postprocess_batch → per-image face restore/overlay/color-correct → images.save_image → grid assembly → extra_networks.deactivate → Processed.
- Extra-network data flows prompt → p.parse_extra_network_prompts() (extra_networks.parse_prompts strips <name:args> tags) → p.extra_network_data dict → extra_networks.activate(p, data) which calls each registered ExtraNetwork.activate (Lora patches model weights, ExtraNetworkHypernet applies hypernets), deactivated after the batch loop.
- Prompt → conditioning: setup_conds() uses prompt_parser.get_learned_conditioning / get_multicond_learned_conditioning to build ScheduledPromptConditioning lists and MulticondLearnedConditioning, which sd_samplers_cfg_denoiser recombines per step; p.c/p.uc cache positive/negative conditioning (cached_c/cached_uc).
- Latents → pixels: p.sample() returns latent samples; if not already_decoded they are NaN-tested (devices.test_for_nans) then decoded via decode_latent_batch (full VAE or opts.sd_vae_decode_method approximations), converted to uint8 numpy then PIL Image.
- Image persistence: images.save_image(image, p.outpath_samples, ...) writes PNG with infotext in pnginfo 'parameters' chunk (or EXIF for jpg/webp), firing script_callbacks.before_image_saved/image_saved; filenames derive from opts.samples_filename_pattern; the first batch's infotext is also written to data_path/params.txt unless --no-prompt-history.
- Processed object is the output boundary: fields images, infotexts, info, seed, index_of_first_image; UI returns processed.images + processed.js() (JSON for JS gallery metadata); API base64-encodes images via encode_pil_to_base64 preserving PNG metadata/EXIF.
- Settings persistence: Options.save() writes JSON to shared.config_filename on any UI settings change; disabled_extensions/disable_all_extensions live in that file and are read back by launch_utils.list_extensions() at next boot; corrupt configs are quarantined to tmp/config.json.
- Checkpoint identity flows through sha256 hashes computed in modules/hashes.py (cached in cache.json), producing CheckpointInfo.hash/short sha256 used in infotexts ('Model hash'), extra-network metadata, and the API /sdapi/v1/sd-models listing.
- Shared mutable state coordination: shared_state.State owns job_count/job/interrupted/skipped/stopping_generation and server command queue (wait_for_server_command/request_restart); progress.start_task/finish_task and TotalTQDM update the console and /internal/progress endpoint.
Behavior traces
Three flows were traced end-to-end. Startup is a two-stage boot: launch.py environment prep, then webui.py UI/API bring-up with a restartable inner loop. txt2img generation flows UI/API → processing dataclass → script-runner hooks → sampler → VAE decode → save. Extension install flows Extensions tab → git clone → arbitrary install.py execution → rescan.
- STARTUP (shell): webui.sh sources webui-user.sh (and webui-macos-env.sh on darwin), creates/activates venv unless venv_dir='-', then execs 'python launch.py'; launch.main() → launch_utils.prepare_environment() → (pip installs, git clones, run_extensions_installers runs each extension's install.py) → start() → import webui → webui.webui().
- STARTUP (python): webui.webui() → initialize.initialize() → initialize_util fixups → sd_models.setup_model() → codeformer_model.setup_model/gfpgan_model.setup_model → initialize_rest(): sd_samplers.set_samplers → extensions.list_extensions → restore_config_state_file → sd_models.list_models → localization.list_localizations → scripts.load_scripts (exec_modules every scripts/*.py and extension scripts) → modelloader.load_upscalers → sd_vae.refresh_vae_list → sd_hijack.list_optimizers → sd_unet.list_unets → background Thread(load_model) → ui_extra_networks.initialize/register_default_pages → extra_networks.register_default_extra_networks.
- STARTUP (server): ui.create_ui() builds all gr.Blocks → shared.demo.queue(64) → demo.launch(server_name/port/share/auth/tls/subpath) → CORS middleware stripped (app.user_middleware filtered to remove CORSMiddleware, comment cites RyotaK attack) → initialize_util.setup_middleware → progress.setup_progress_api → ui.setup_ui_api → create_api(app) if --api → ui_extra_networks.add_pages_to_demo → app_started_callback → wait_for_server_command loop.
- TXT2IMG (UI): Generate button → wrap_gradio_gpu_call (acquires queue_lock, state.begin, progress.start_task) → txt2img() → txt2img_create_processing() → p.scripts.run (returns None if no selectable script) → processing.process_images(p) → processed.js() returned to gallery/HTML outputs; errors are caught into an HTML error div by wrap_gradio_call.
- TXT2IMG (inner): process_images() → scripts.before_process → apply override_settings → process_images_inner() → p.init(all_prompts,seeds,subseeds) → loop n_iter: before_process_batch → parse_extra_network_prompts → extra_networks.activate → process_batch → setup_conds → write params.txt (n==0) → p.sample() [Txt2Img: sampler.sample → optional sample_hr_pass for hires fix] → post_sample → decode_latent_batch → postprocess_batch/postprocess_batch_list → per-image postprocess_image → save_image → image_grid → postprocess → Processed.
- IMG2IMG/INPAINT: img2img() collects init_images/mask/sketch from the six sub-tabs → StableDiffusionProcessingImg2Img.init() encodes init image to latent (latent_image), builds mask (image_mask/inpainting conditioning), p.sample() runs sampler at denoising_strength with img2img_fix_steps; on_mask_blend/postprocess_maskoverlay hooks fire for inpaint compositing before save.
- API GENERATION: POST /sdapi/v1/txt2img → Api.text2imgapi (behind Depends(self.auth) if --api-auth) → apply_infotext fills unset request fields → get_selectable_script resolves script_name → validate_sampler_name → init_script_args builds flat args (index 0 = selectable script idx+1) → with queue_lock: StableDiffusionProcessingTxt2Img(**args) → p.scripts.run or process_images → base64 images + info JSON in TextToImageResponse.
- EXTENSION INSTALL: Extensions tab → install_extension_from_url(dirname,url) → check_access() asserts not cmd_opts.disable_extension_access → git.Repo.clone_from(url, tmpdir, filter=['blob:none']) + submodule.update → os.rename/shutil.move to extensions_dir → launch.run_extension_installer executes the extension's install.py → extensions.list_extensions() rescans; apply_and_restart triggers restart_program/stop_program.
- SETTINGS WRITE: UI settings component change → shared.opts.<key> = value → Options.__setattr__ validates against freeze-settings flags, restricted component_args, frozen sections/keys → data[key]=value → onchange callbacks via opts.set() → opts.save(config_filename) writes config.json; at boot Options.load() runs version migrations (sd_vae_as_default→sd_vae_overrides_per_model_preferences, quicksettings→quicksettings_list, ui_reorder→ui_reorder_list).
- MODEL RELOAD: override_settings.sd_model_checkpoint or /sdapi/v1/reload-checkpoint → sd_models.reload_model_weights → get_checkpoint_state_dict (safetensors.load_file or monkeypatched torch.load via safe.load) → send_model_to_trash old model → load_model_weights → LoadStateDictOnMeta streams weights → send_model_to_device → model_hijack.hijack + embedding_db.load_textual_inversion_embeddings(force_reload).
Testing signals
Testing is API-level only: pytest + requests against a live server started with --test-server on CPU using test/test_files/empty.pt as the checkpoint. test/ holds 6 test modules covering txt2img, img2img, extras, face restorers, torch utils, and endpoint smoke tests. CI (run_tests.yaml) launches the real server on ubuntu-latest with coverage, runs pytest with pytest-base-url, then kills it via the /sdapi/v1/server-stop route. Lint CI runs ruff (pinned 0.3.3) and eslint. There are no unit tests of the sampling pipeline, no UI tests, and coverage of modules/ internals is incidental.
- Test framework is pytest 7.3 + pytest-base-url + pytest-cov (requirements-test.txt); pyproject.toml [tool.pytest.ini_options] sets base_url=http://127.0.0.1:7860; tests use plain requests calls, no fixtures beyond conftest base64 image encoders and an 'initialize' fixture that imports webui.
- Tests are end-to-end HTTP tests: test_txt2img.py POSTs a minimal 64x64 3-step 'Euler a' request to /sdapi/v1/txt2img; test_img2img.py covers img2img/inpaint modes with base64 fixtures; test_utils.py smoke-tests GET endpoints (samplers, sd-models, embeddings, ...) and a read-write /sdapi/v1/options roundtrip.
- CI runs the actual app: run_tests.yaml does 'python launch.py --skip-torch-cuda-test --exit' to prepare env, then starts 'launch.py --test-server --use-cpu all --api-server-stop' under coverage, waits on port 7860, runs pytest --verify-base-url test, and stops the server via curl -XPOST /sdapi/v1/server-stop.
- --test-server mode mutates sys.argv to inject --api, --ckpt test/test_files/empty.pt, --skip-torch-cuda-test, --disable-nan-check and blanks COMMANDLINE_ARGS, letting CI run generation-shaped calls without a real model or GPU.
- conftest.py sets IGNORE_CMD_ARGS_ERRORS=1 so pytest's own argv doesn't break shared_cmd_options' argparse parse (which otherwise calls parser.parse_args() at import time).
- Lint coverage: on_pull_request.yaml runs ruff==0.3.3 (config in pyproject.toml, target py39, selects B/C/I/W, excludes extensions/) and eslint via 'npm run lint' (eslint ^8.40.0, scripts lint/fix in package.json).
- Coverage artifacts are produced (coverage combine + html uploaded as artifacts), but nothing enforces a coverage floor; the core diff math — sd_samplers_kdiffusion, sd_samplers_cfg_denoiser, prompt_parser, sd_hijack — has no direct unit tests and is only exercised indirectly through the 6 API test files. (inferred)
- test/test_files contains tiny fixtures (empty.pt fake checkpoint, img2img_basic.png, mask_basic.png, two-faces.jpg); test_outputs/ is an output scratch dir — there is no golden-image regression comparison of generated outputs.
Risk map
The dominant risk theme is remote-code-execution surface by design: extensions are arbitrary Python loaded at startup, the Extensions tab can git-clone and run install.py, and the Custom code script exec()s user code — mitigated only by flags that auto-disable extension access when the server is non-local. The API is unauthenticated unless --api-auth is set and can fetch remote URLs (SSRF-checked but opt-out-able). Shell-out is pervasive in launch_utils (shell=True subprocess). Checkpoint loading relies on a monkeypatched torch.load pickle scan that --disable-safe-unpickle bypasses.
- Arbitrary code execution by design: every .py in extensions/*/scripts and extensions/*/preload.py is exec_module'd at startup, and install_extension_from_url runs the extension's install.py; protection is only cmd_opts.disable_extension_access = webui_is_non_local AND not --enable-insecure-extension-access, asserted by check_access().
- scripts/custom_code.py executes arbitrary Python via exec()/eval() (exec_with_return builds an AST and runs it) gated only by --allow-code flag and a re-check inside run().
- The REST API has no auth by default: --api-auth is optional HTTP Basic; without it every /sdapi/v1/* route (including POST /options which can change output dirs, /train/* jobs, and --api-server-stop's /server-kill|restart|stop) is open to anyone who can reach the port.
- SSRF surface: decode_base64_to_image fetches http(s) URLs server-side when opts.api_enable_requests; verify_url() checks resolved IPs are global (blocks local addresses only when api_forbid_local_requests), but DNS-rebinding between gethostbyname_ex and requests.get is not closed.
- shell=True subprocess everywhere in launch_utils.run(): pip installs, git clone/pull/fetch/reset built as f-strings; git_clone interpolates URL/dir into a shell string (quoted but not escaped); TORCH_COMMAND/INDEX_URL env vars directly shape pip commands.
- Checkpoint deserialization: torch.load is globally replaced by safe.load which scans pickles (check_pt) for malicious code before unsafe_torch_load; --disable-safe-unpickle turns this off entirely; the scan is a blocklist-style heuristic, and safetensors path bypasses it.
- Gradio's permissive CORS middleware is manually stripped post-launch (app.user_middleware filter) specifically because a malicious page could otherwise drive the local API to install extensions; any code re-adding middleware could silently reopen this hole.
- File-system writes outside repo root: --data-dir/--ckpt-dir/--embeddings-dir and outdir_* options let users/API redirect outputs and model loads to arbitrary paths; images.save_image and Options.save have no sandboxing beyond --gradio-allowed-path for serving and --hide-ui-dir-config for hiding dir settings.
- Single global queue_lock + mutable shared.sd_model/shared.opts: process_images mutates global opts via override_settings then restores in finally — an exception or crash between set and restore, or concurrent API calls interleaving opts changes, can leave settings or the loaded checkpoint in an inconsistent state. (inferred)
- Options.__setattr__ uses assert for freeze-settings enforcement ('assert not cmd_opts.freeze_settings'); running Python with -O would strip these checks, silently disabling the freeze protections.
- Startup update path fetches remote state and prints notices (version_check hits api.github.com; --update-all-extensions runs git pull --autostash across every repo under extensions_dir including subrepos) — network-dependent and can leave extension trees in autostash-conflict states.
- sys.path is temporarily prepended with each script's basedir during load_scripts() — an extension script can shadow stdlib/modules imports for subsequently loaded scripts (restored in finally per file, but cross-file ordering via metadata.ini Requires/Before/After is attacker-orderable). (inferred)
Extension points
This repo is primarily an extension host. Four plug-in surfaces exist: (1) extensions in extensions/ and extensions-builtin/ — directories with optional preload.py (argparse), install.py (setup), scripts/*.py (Script subclasses), metadata.ini (ordering/deps), and javascript/*.js; (2) the Script base class lifecycle with ~20 hooks from before_process to postprocess; (3) the script_callbacks.py registry with ~20 named callback categories including CFG denoiser per-step hooks; (4) registries for extra networks, samplers, upscalers, optimizers, and UNET patches.
- Extension discovery: extensions.list_extensions() scans extensions-builtin/ then extensions/, building Extension objects gated by --disable-all-extensions/--disable-extra-extensions and opts.disabled_extensions; ExtensionMetadata reads metadata.ini for canonical Name and [Extension] Requires plus per-script Requires/Before/After ordering fields.
- Script loading: scripts.load_scripts() runs list_scripts('scripts','.py') + list_scripts('modules/processing_scripts','.py') over the base dir and every active extension, orders files by topological_sort of metadata dependencies, load_module() exec's each file, and registers every Script or ScriptPostprocessing subclass found into scripts_data/postprocessing_scripts_data.
- Script API: subclass modules.scripts.Script, implement title()/ui(is_img2img)/show()/run(p,*args); AlwaysVisible return from show() makes it always-on; lifecycle hooks include setup, before_process, process, before_process_batch, after_extra_networks_activate, process_batch, postprocess_batch, postprocess_batch_list, on_mask_blend, post_sample, postprocess_image, postprocess_image_after_composite, postprocess, before_hr — each documented in the class.
- script_callbacks.py exposes registration functions on_app_started, on_before_reload, on_model_loaded, on_ui_tabs, on_ui_train_tabs, on_ui_settings, on_before_image_saved, on_image_saved, on_extra_noise, on_cfg_denoiser, on_cfg_denoised, on_cfg_after_cfg, on_before_component, on_after_component, on_image_grid, on_infotext_pasted, on_script_unloaded, on_before_ui, on_list_optimizers, on_list_unets, on_before_token_counter — backed by callback_map lists with ordered sorting and per-extension callback naming.
- on_ui_tabs is how extensions add top-level Gradio tabs: callback returns [(blocks, label, elem_id)] tuples appended to interfaces in create_ui(); on_before_component/after_component (or Script.on_before_component(elem_id=...)) lets scripts inject/modify any Gradio component by elem_id.
- Extra networks: call extra_networks.register_extra_network(obj) (name matches <name:...> in prompts) and register_extra_network_alias; obj implements activate(p, params_list)/deactivate(p). extensions-builtin/Lora registers ExtraNetworkLora with alias 'lyco' in its scripts/lora_script.py before_ui callback.
- preload.py hook: any extension dir containing preload.py gets load_module'd and its preload(parser) called during shared_cmd_options import, letting extensions add CLI args before cmd_opts is parsed (Lora uses it for --lora-dir/--lyco-dir-backcompat).
- Settings extension: update shared.options_templates with shared.options_section(...) dicts of OptionInfo — Lora adds 'sd_lora', 'lora_*' options this way; each OptionInfo can carry an onchange callback and restrict_api flag.
- Extra Networks pages: ui_extra_networks.register_page(ExtraNetworksPage subclass) adds browsable card tabs; ui_extra_networks_user_metadata.py shows per-file .json sidecar editing — pages are served to the browser via add_pages_to_demo.
- Sampler/scheduler/upscaler registration: append SamplerData to sd_samplers_*.py lists consumed by all_samplers; sd_schedulers.schedulers; upscaler.Upscaler subclasses discovered by modelloader.load_upscalers() scanning models dirs; optimizers via on_list_optimizers into sd_hijack_optimizations; UNET patches via on_list_unets into sd_unet.
- API surface for scripts is automatic: Api.__init__ initializes scripts_txt2img/img2img runners if empty, builds default_script_arg arrays from each script's ui() default component values, and get_script_info exposes Script.api_info — scripts appear in /sdapi/v1/scripts and are addressable by name in requests' script_name/alwayson_scripts.
- Built-in extensions in extensions-builtin/ (canvas-zoom-and-pan, extra-options-section, hypertile, LDSR, Lora, mobile, postprocessing-for-training, prompt-bracket-checker, ScuNET, soft-inpainting, SwinIR) are the reference implementations for each extension pattern — is_builtin=True keeps them loaded even under --disable-extra-extensions.
Contributor handoff
Read webui.py, modules/initialize.py, modules/processing.py, modules/scripts.py, modules/shared.py, and modules/paths_internal.py first — they define the boot order, the p-object pipeline, and the global-state conventions everything else hangs off. Safe edit zones: scripts/ and extensions-builtin/ for new features; modules/api/api.py + modules/api/models.py for new endpoints; shared_options.py for new settings. Traps: import-order sensitivity (paths_internal before torch), the single queue_lock serialization, opts restore semantics, and test mode requiring --test-server conventions.
- Import-order discipline matters: modules/paths_internal.py and modules/cmd_args.py parse sys.argv at import time and inject COMMANDLINE_ARGS; shared_cmd_options imports launch.py and runs extension preload.py files at import — never import torch-dependent modules before initialize.imports(), and set IGNORE_CMD_ARGS_ERRORS=1 when running pytest/other tools.
- Follow the 'p' convention: generation features should be implemented as Script subclasses (scripts/ or extension scripts/) mutating the StableDiffusionProcessing object via the documented hook lifecycle, not by editing process_images_inner directly — the base class docstrings define exactly when each hook fires.
- Adding a REST endpoint: add a pydantic model in modules/api/models.py, register with self.add_api_route in Api.__init__ (auto-wraps Depends(auth) when --api-auth is set), reuse init_script_args/apply_infotext for generation endpoints, and wrap generation calls in self.queue_lock.
- Adding a setting: add an OptionInfo entry to the right options_section in modules/shared_options.py (or shared.options_templates.update from an extension); set restrict_api=True to hide it from /sdapi/v1/options; onchange callbacks run via opts.set(); do_not_save entries never persist.
- Code style is ruff-governed (pyproject.toml: target py39, select B/C/I/W, E501/E721/E731/C901 ignored, extensions/ excluded); run 'ruff .' and 'npm run lint' before committing; CI also requires PRs from forks (lint/test jobs skip same-repo PRs).
- To run tests locally you must boot the app first: 'python launch.py --test-server --skip-prepare-environment --skip-torch-cuda-test --use-cpu all --api-server-stop' then 'pytest test' — tests hit a live server on :7860, there is no offline unit-test path for the pipeline.
- Beware mutable module globals: shared.sd_model may be swapped mid-generation by refiner/override_settings (reload_model_weights is called inside the batch loop); shared.opts changes during a job are restored by process_images' finally block only if override_settings_restore_afterwards — keep new opts writes inside that contract.
- Infotext is the cross-cutting serialization format: paste_fields in modules/infotext_utils.py map infotext labels ↔ gradio components ↔ API fields; any new generation parameter must be added there (and to create_infotext's generation_params dict) to roundtrip through PNG metadata and 'Send to' buttons.
- File-not-found and extension errors should go through modules/errors.py report()/display() (they render in UI and log), not bare exceptions; scripts' callback failures are individually caught by script_callbacks report_exception so one bad extension doesn't kill the run.
- Device/memory plumbing is centralized in modules/devices.py (autocast, torch_gc, test_for_nans, dtype selection via --no-half/--precision/--use-cpu) and modules/lowvram.py (--lowvram/--medvram model shuffling) — new GPU work should reuse devices.get_optimal_device/autocast/without_autocast contexts rather than raw torch calls.
Languages
| Language | Files | Share of source |
|---|---|---|
| Python | 213 | 86% |
| JavaScript | 30 | 12% |
| YAML | 14 | docs/config |
| HTML | 11 | docs/config |
| Markdown | 3 | docs/config |
| CSS | 2 | docs/config |
| Shell | 2 | 1% |
| TOML | 2 | 1% |
Start reading here
Probable source entrypoint: pyproject.toml — inferred from the directory tree and manifest, not the upstream docs.
Source roots
modules/ (152 files) extensions-builtin/ (43 files) javascript/ (25 files)
Source layout — modules/
| Path | Files |
|---|---|
modules/api/ | 2 |
modules/hypernetworks/ | 2 |
modules/models/ | 9 |
modules/processing_scripts/ | 4 |
modules/textual_inversion/ | 7 |
modules/cache.py | 1 |
modules/call_queue.py | 1 |
modules/cmd_args.py | 1 |
modules/codeformer_model.py | 1 |
modules/config_states.py | 1 |
modules/dat_model.py | 1 |
modules/deepbooru.py | 1 |
modules/deepbooru_model.py | 1 |
modules/devices.py | 1 |
Repository structure
| Top-level path | Files |
|---|---|
modules/ | 152 |
extensions-builtin/ | 43 |
javascript/ | 25 |
scripts/ | 12 |
html/ | 11 |
test/ | 9 |
.github/ | 7 |
configs/ | 7 |
Docs found: README.md
Package
package.json — stable-diffusion-webui v0.0.0, 1 declared dependencies.
eslint
CI & config entrypoints
README.md— Repository readme.github/workflows/warns_merge_master.yml— GitHub Actions workflowpackage.json— npm scripts: fix, lint
Structural health signals
Average score 8.3/10 across 96 analyzed source files; 3 hotspots surfaced.
extensions-builtin/canvas-zoom-and-pan/javascript/zoom.js— 995 lines, score 6.2 (Large file, Deep nesting)modules/launch_utils.py— 482 lines, score 6.4 (Large file, Deep nesting)modules/ui_extensions.py— 701 lines, score 6.4 (Large file, Deep nesting)
These are deterministic review leads — size, churn, and structural signals — not proof of a bug in this project.
Recent history
82a973c043672024-07-27 — changelog1d7e9eca098d2024-07-27 — Merge pull request #16275 from AUTOMATIC1111/fix-image-upscale-on-cpuc19d044364962024-07-27 — Merge branch 'release_candidate'8b3d98c5a5802024-07-20 — update CHANGELOG5bbbda473fc82024-07-20 — Merge branch 'dev' into release_candidate