ollama/ollama

Single Go module (github.com/ollama/ollama) whose main binary is both the CLI client and the server. `ollama serve` starts a Gin HTTP server that owns a model Scheduler; inference is delegated to subprocesses (llama-server for GGUF, `ollama runner` for MLX/safetensors) spoken to over loopback HTTP. Model artifacts live as sha256 blobs + JSON manifests under OLLAMA_MODELS. Native code (vendored llama.cpp under llama/, MLX under mlx/) is built by CMake; the desktop app adds a webview shell and a React/Vite UI. Everything is configured through OLLAMA_* environment variables centralized in envconfig.

• Process entrypoint is main.go which calls cmd.NewCLI().ExecuteContext; all user-facing commands are cobra subcommands registered in NewCLI in cmd/cmd.go
  main.gocmd/cmd.go#L2385-L2600
• The HTTP server is a Gin engine built by Server.GenerateRoutes, registering ~30 routes across /api/*, OpenAI-compatible /v1/*, and Anthropic /v1/messages
  server/routes.go#L1852-L1949
• Inference scheduling is a channel-based Scheduler with pendingReqCh/finishedReqCh/expiredCh/unloadedCh, a loaded map of runnerRef keyed by model path, and a single in-flight load (activeLoading)
  server/sched.go#L61-L106
• GGUF inference runs in a separate llama-server subprocess spawned via exec.Command and driven over loopback HTTP through the llm.LlamaServer interface (Completion, Chat, Embedding, Tokenize, WaitUntilRunning, Close)
  llm/llama_server.go#L120-L160llm/llama_server.go#L424llm/server.go#L58-L77
• Safetensors/MLX models take a different runner path: sched.load detects m.IsMLX() (Config.ModelFormat == "safetensors") and creates an mlxrunner.Client which re-execs the same binary as `ollama runner --model X --port N`
  server/sched.go#L530-L594server/images.go#L93-L95mlxrunner/client.go#L349cmd/cmd.go#L2569-L2576
• Model storage is content-addressed: manifest.Path() -> OLLAMA_MODELS/manifests/<name>, BlobsPath() -> OLLAMA_MODELS/blobs/sha256-<hex>; manifests map model names to typed layers
  manifest/paths.go#L29-L72server/model.go#L27-L97
• Configuration is centralized in envconfig as OLLAMA_* variables (HOST, MODELS, KEEP_ALIVE, NUM_PARALLEL, MAX_LOADED_MODELS, MAX_QUEUE, ORIGINS, FLASH_ATTENTION, KV_CACHE_TYPE, NO_CLOUD, REMOTES, etc.) exposed via AsMap()
  envconfig/config.go#L20-L341
• The desktop app is a separate main package gated by `//go:build windows || darwin` in app/cmd/app/app.go, bundling a webview, sqlite-backed store (app/store/schema.sql), updater, and a React 19 + Vite + Tailwind UI under app/ui/app
  app/cmd/app/app.go#L1-L34app/store/schema.sqlapp/ui/app/package.json
• Full build is CMake-driven (cmake -B build .; cmake --build build) producing the Go binary plus native payload under build/lib/ollama; go build ./go run . serve works for Go-only iteration against an existing native payload
  AGENTS.mddocs/development.mdCMakeLists.txt
• Native engines are vendored/pinned: llama.cpp version in LLAMA_CPP_VERSION with llama/server sources, MLX pinned via MLX_VERSION and MLX_C_VERSION under mlx/, GPU backends selected via -DOLLAMA_LLAMA_BACKENDS=cuda_v12;cuda_v13;rocm_v7_1;rocm_v7_2;vulkan;cuda_jetpack5;cuda_jetpack6
  docs/development.md.github/workflows/test.yaml#L40-L49LLAMA_CPP_VERSIONMLX_VERSION
• GPU/device discovery is performed by spawning helper binaries: `ollama gpu-discover` (hidden cobra cmd -> discover.RunNativeProbeCommand) and llama-server --list-devices probes in discover/llama_server.go
  cmd/cmd.go#L2581-L2590discover/native_probe.go#L86discover/llama_server.go#L143discover/runner.go#L36-L80
• The CLI talks to the server through api.Client (api/client.go) whose methods map 1:1 to the REST API and which reads OLLAMA_HOST via ClientFromEnvironment
  api/client.go#L1-L79