CodeVetter · Repo Unpack · Testing signals
langchain-ai/langchain
pytest is the framework everywhere, configured per-package with strict markers/config, asyncio auto mode, syrupy snapshots, and pytest-socket to forbid network in unit tests. langchain_v1 has ~85 unit-test files (heavy agents/middleware coverage, plus chat_models, embeddings, mcp, tools), integration tests with VCR cassettes, and a benchmark suite for create_agent. CI runs tests twice — current and minimum dependency versions — and scheduled integration tests trace to LangSmith. langchain-tests packages the conformance suite partners must pass. Gaps: unit tests need docker postgres/redis for checkpointer paths, and langchain-classic's huge legacy surface relies on older coverage.
- pytest config: `--strict-markers --strict-config --durations=5 --snapshot-warn-unused -vv`, asyncio_mode=auto, custom markers (requires, scheduled, compile, benchmark), syrupy snapshots under tests/**/__snapshots__.
- Unit tests live in tests/unit_tests mirroring source layout; langchain_v1's agents suite includes test_react_agent.py, test_response_format*.py, test_return_direct*.py, test_subagent_*.py, middleware/ and middleware_typing/ dirs, plus specifications/ and cassettes/ for VCR replay.
- Network isolation is enforced: `make test` runs pytest with `--disable-socket --allow-unix-socket` (pytest-socket), and unit tests must not hit the network per AGENTS.md.
- Persistence tests are not hermetic: `make test`/`extended_tests`/`test_watch` spin up postgres and redis via `tests/unit_tests/agents/compose-postgres.yml` and `compose-redis.yml` before running, then tear them down.
- Async correctness is checked with blockbuster (blocking-call detection) in the test group; freezegun and responses are used in core tests.
- Benchmarks exist for agent creation: `make benchmark` runs tests/benchmarks/test_create_agent.py with -m benchmark; a codspeed workflow exists for perf tracking.
- CI runs the unit suite twice — once on current deps and once after installing computed minimum versions via `.github/scripts/get_min_versions.py` — catching lower-bound regressions.
- Scheduled integration tests (integration_tests.yml) trace runs to LangSmith using LANGSMITH_TRACING/API_KEY/PROJECT/TAGS env vars and authenticate to cloud providers via workload identity (google-github-actions/auth, configure-aws-credentials).
- langchain-tests is a published package exposing standard unit+integration test classes (chat_models, embeddings, tools, vectorstores, retrievers, indexers, caches, sandboxes) that partner packages subclass for conformance.
- Static checks are strict: ruff with `select = ["ALL"]` (google docstring convention, banned relative imports) and mypy strict mode with pydantic plugin; `check_imports` script verifies every module imports cleanly in CI.
- Coverage is gated by make targets `coverage` and `coverage_agents` (term-missing + xml/html reports scoped to langchain.agents); test_react_agent.py is exempted from all lint rules, suggesting it is a large legacy-style test file.