CodeVetter · Repo Unpack · System map
langgenius/dify
The system is a modular monolith API (Flask + flask-restx + SQLAlchemy + Celery) plus a Next.js console, with execution-heavy work pushed to Celery workers and sidecar services (plugin daemon, sandbox, agent runtime, SSRF proxy). `api/app.py` is the process entrypoint; `app_factory.create_app()` builds a `DifyApp`, runs an ordered list of ~30 `extensions/ext_*` initializers, and wraps the result in `socketio.WSGIApp`. HTTP ingress is split across eight Flask blueprints registered in `ext_blueprints.py`, each with its own CORS/auth surface. State lives in Postgres (Alembic, 219 migration files), Redis (broker, cache, rate limits, pub/sub for SSE), pluggable object storage, and pluggable vector DBs loaded via Python entry points.
- `api/app.py` is the API process entrypoint: when run as `python -m app` it gevent-monkey-patches stdlib, grpc, and psycopg2 BEFORE other imports, then serves the socketio-wrapped app on 0.0.0.0:5001 via `gevent.pywsgi.WSGIServer` with `WebSocketHandler`.
- `app_factory.create_app()` returns a `(socketio.WSGIApp, DifyApp)` pair; `initialize_extensions()` iterates a fixed ordered list of ~30 `extensions/ext_*` modules (timezone → logging → database → redis → storage → key_provider → celery → login → blueprints → otel …) calling `init_app(app)` on each, skipping any ext exposing `is_enabled() == False`.
- All HTTP surfaces are Flask blueprints registered in `ext_blueprints.init_app`: `/v1` service_api, `/api` web (split CORS: unauthenticated embed `/chat-messages*` vs authenticated default), `/console/api`, `/files`, `/inner/api`, `/mcp`, `/triggers`, and an `/openapi` blueprint gated behind `dify_config.OPENAPI_ENABLED`.
- `flask db` commands get a deliberately reduced app: `is_db_command()` detects the argv shape and `create_migrations_app()` initializes only `ext_database`, `ext_migrate`, and `ext_commands` — Alembic over `api/migrations` (219 version files).
- The Celery worker entrypoint `celery_entrypoint.py` applies the same gevent/grpc/psycogreen patching then imports `app, celery`; `ext_celery.init_app` creates a Celery app with a `FlaskTask` base class that wraps every task in `app.app_context()` plus request logging context, and stores it in `app.extensions["celery"]`.
- Configuration is a single composed `DifyConfig` pydantic-settings class built from mixins (PackagingInfo, DeploymentConfig, FeatureConfig, MiddlewareConfig, ExtraServiceConfig, ObservabilityConfig, RemoteSettingsSourceConfig, EnterpriseFeature/TelemetryConfig) with source precedence init > env > remote-settings > .env > file secrets > pyproject TOML.
- Remote configuration is an extension point: `RemoteSettingsSourceFactory` reads `REMOTE_SETTINGS_SOURCE_NAME` and delegates to `ApolloSettingsSource` or `NacosSettingsSource` to hydrate every model field.
- The Flask app object is a `DifyApp` subclass carrying typed extension attributes (e.g. `login_manager: DifyLoginManager`); config is pushed into `app.config` from `dify_config.model_dump()` and `RESTX_ERROR_404_HELP` is disabled so flask-restx 404s don't enumerate routes to anonymous callers.
- Object storage is abstracted by `extensions/ext_storage.py`: `Storage.get_storage_factory` matches `STORAGE_TYPE` to per-provider runners (S3, OpenDAL, local fs, Azure Blob, Aliyun OSS, GCS, Tencent COS, Oracle OCI, Huawei OBS, Baidu OBS, Supabase, TOS, …) implemented under `api/extensions/storage/`.
- Vector DBs are hollow workspace packages under `api/providers/vdb/vdb-*` that register `dify.vector_backends` Python entry points (e.g. vdb-pgvector → `PGVectorFactory`); ~28 backends exist and `api/pyproject.toml` defines per-backend uv dependency groups (`vdb-all`, `vdb-qdrant`, …) plus a parallel `providers/trace/*` workspace for tracing backends.
- The graph execution engine is an external pinned dependency `graphon==0.7.0` (not in-repo): `WorkflowEntry` composes `graphon.Graph`, `GraphEngine`, `VariablePool`, `GraphRuntimeState`, command channels, and engine layers; Dify-owned nodes under `core/workflow/nodes/` cover agent, agent_v2, datasource, human_input, knowledge_index/retrieval, and trigger_* types.
- Deployment is a generated `docker/docker-compose.yaml` (regenerated by `docker/generate_docker_compose` from `docker-compose-template.yaml` + envs) running api, worker, worker_beat, scheduler, web, postgres, mysql, redis, `dify-sandbox:0.2.15`, `dify-plugin-daemon:0.6.10-local`, `dify-agent-backend`, `dify-agent-local-sandbox`, two Squid SSRF proxies, certbot and nginx.
- The web tier is a Next.js standalone build (`output: 'standalone'`, `typescript.ignoreBuildErrors: true`) using Turbopack rules, `transpilePackages` for echarts/zrender/env libs, and `serverExternalPackages: ['loro-crdt']`; base path and dev origins come from `web/env.ts`.
- The frontend talks to the console API through a generated oRPC client: `web/service/console/index.ts` builds `consoleClient`/`consoleQuery` over `consoleRouterContract` from `@dify/contracts`, with a `DynamicLink` switching between browser transport and a server-side link registered on `globalThis.$consoleServerLink`.
- Backend↔plugin-daemon traffic is plain HTTP: `core/plugin/impl/base.py` `BasePluginClient` issues requests against `PLUGIN_DAEMON_URL`/`PLUGIN_DAEMON_KEY` config; specialized clients exist per capability (agent, tool, model, datasource, endpoint, oauth, trigger, dynamic_select, debugging, asset).