# Data flow — sqlite/sqlite

> Canonical page: https://codevetter.com/unpack/sqlite/data-flow

# Data flow — sqlite/sqlite

Scanned commit fde3a84dddf35f4c19797c8afad04a44c596f4d8 (2026-09-21). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Data enters as UTF-8 (or UTF-16, transcoded at the API edge) SQL text or as bound parameter values. `sqlite3RunParser` drives `sqlite3GetToken` token-by-token into the Lemon parser, which builds `Select`/`Expr`/`SrcList` trees inside a `Parse` object; the code generators turn those trees into a VDBE program of `Op` records. At run time the VDBE owns register memory (`Mem *aMem`) and cursors; reads and writes go through B-Tree cursors into the pager's page cache, and out to disk either as rollback-journal writes or WAL frame appends, always via `sqlite3_vfs` methods. Output leaves either as `sqlite3_column_*` values pulled from `p->pResultRow`, or — in the CLI — through QRF's `xWrite` callback into a `FILE*`. Schema itself round-trips through this same machinery: `sqlite3_schema` rows are re-parsed on load, which is why a schema cookie mismatch forces `sqlite3Reprepare`.
- Text input is bounded at the tokenizer: `sqlite3RunParser` decrements `mxSqlLen` (initialized from `db->aLimit[SQLITE_LIMIT_SQL_LENGTH]`) by each token length and aborts with `SQLITE_TOOBIG` when it goes negative. — [src/tokenize.c#L600-L655](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/tokenize.c#L600-L655)
- The tokenizer→parser handoff is a push loop: `sqlite3GetToken()` returns a token type and length, context-sensitive keywords are disambiguated by `analyzeWindowKeyword`/`analyzeOverKeyword`/`analyzeFilterKeyword`, and each token is pushed into the Lemon engine with `sqlite3Parser(pEngine, tokenType, pParse->sLastToken)`. — [src/tokenize.c#L644-L715](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/tokenize.c#L644-L715)
- At end of input the parser is deliberately fed two synthetic tokens — `TK_SEMI` then `0` — so the grammar can reduce cleanly without a trailing semicolon in the source text. — [src/tokenize.c#L672-L684](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/tokenize.c#L672-L684)
- SQL comments are conditionally dropped in the token stream rather than the grammar: `TK_COMMENT` is skipped only when `db->init.busy` (schema reparse) or `SQLITE_DBCONFIG_ENABLE_COMMENTS` is on, otherwise it falls through to an 'unrecognized token' error. — [src/tokenize.c#L697-L706](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/tokenize.c#L697-L706)
- UTF-16 inputs are normalized at the boundary: `sqlite3_open16` converts the filename via `sqlite3ValueText(pVal, SQLITE_UTF8)` before calling `openDatabase`, and sets `SCHEMA_ENC`/`ENC` to `SQLITE_UTF16NATIVE` if the schema has not yet loaded. — [src/main.c#L3773-L3805](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/main.c#L3773-L3805)
- Query plans are produced by `sqlite3WhereBegin()`, which chooses the nesting order of FROM-clause tables based on available indexes rather than written order, and emits the cursor-opening code that `sqlite3WhereEnd()` later closes. — [src/where.c#L6770-L6845](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/where.c#L6770-L6845) [src/where.c#L7541](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/where.c#L7541)
- `sqlite3Select()` is the funnel for all query code generation; it takes a `SelectDest` describing where rows go (`SRT_Output`, `SRT_Exists`, `SRT_Fifo`, `SRT_DistQueue`, …), so the same tree compiler serves top-level queries, subqueries, and recursive CTE queues. — [src/select.c#L7680-L7735](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/select.c#L7680-L7735)
- Dirty-page flow at commit is branch-per-journal-mode inside `sqlite3PagerCommitPhaseOne`: in WAL mode it takes `sqlite3PcacheDirtyList()` and calls `pagerWalFrames()` (forcing page 1 into the list if empty, per ticket 2d1a5c67dfc2363e), otherwise it goes down the rollback-journal or batch-atomic-write path. — [src/pager.c#L6551-L6600](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/pager.c#L6551-L6600) [src/pager.c#L3233](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/pager.c#L3233)
- The pager state machine is explicit: `sqlite3PagerCommitPhaseOne` asserts the pager is in `PAGER_WRITER_LOCKED`, `WRITER_CACHEMOD`, `WRITER_DBMOD` or `PAGER_ERROR`, and returns early if it has not reached `PAGER_WRITER_CACHEMOD` (i.e. nothing was modified). — [src/pager.c#L6533-L6552](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/pager.c#L6533-L6552) [doc/pager-invariants.txt](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/doc/pager-invariants.txt)
- WAL readers snapshot through `sqlite3WalBeginReadTransaction` → `walTryBeginRead` → `walIndexReadHdr`, which reads the shared-memory wal-index header; the shm region itself is created by `unixOpenSharedMemory`/`unixLockSharedMemory` in the unix VFS. — [src/wal.c#L3575](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/wal.c#L3575) [src/wal.c#L3102](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/wal.c#L3102) [src/wal.c#L2742](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/wal.c#L2742) [src/os_unix.c#L4850-L5040](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/os_unix.c#L4850-L5040) [doc/wal-lock.md](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/doc/wal-lock.md)
- In the CLI the output boundary is `shell_exec()`: it loops `sqlite3_prepare_v2` over the input buffer using `zLeftover` to advance through multi-statement text, and renders each result set through the QRF spec whose `xWrite` is `shellWriteQR`. — [src/shell.c.in#L3807-L3875](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/shell.c.in#L3807-L3875)
- Interactive input is accumulated line-by-line in `process_input()`, which grows `zSql` by half-again increments, treats a leading `.` or `#` on a fresh buffer as a dot-command, and only executes when a semicolon is present *and* `sqlite3_complete(zSql)` agrees the statement is finished. — [src/shell.c.in#L13473-L13560](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/shell.c.in#L13473-L13560)
- Result rows are owned by the VDBE and invalidated on the next step: `sqlite3Step` returns `SQLITE_ROW` with `p->pResultRow` set, and clears `p->pResultRow = 0` on any non-ROW outcome before running WAL callbacks or transferring errors. — [src/vdbeapi.c#L930-L960](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/vdbeapi.c#L930-L960)
- Error state flows through two channels that must be kept consistent — the per-statement `p->rc` and the per-connection `db->errCode` — reconciled by `sqlite3VdbeTransferError()` only for statements prepared with `SQLITE_PREPARE_SAVESQL`. — [src/vdbeapi.c#L946-L970](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/vdbeapi.c#L946-L970) [src/prepare.c#L955-L972](https://github.com/sqlite/sqlite/blob/fde3a84dddf35f4c19797c8afad04a44c596f4d8/src/prepare.c#L955-L972)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)