# Testing signals — browser-use/browser-use

> Canonical page: https://codevetter.com/unpack/browser-use/testing-signals

# Testing signals — browser-use/browser-use

Scanned commit d8110c5ff87ccba887aaa726cdb780f2f84bef8d (2026-09-15). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

The project uses pytest 9 with pytest-asyncio in auto mode, a 300s default timeout, and xdist. The CI-gated suite is tests/ci, discovered as one GitHub Actions job per test file with a 4 minute job timeout. Tests are expected to use a real browser plus pytest-httpserver; the LLM is the only allowed mock, via conftest.create_mock_llm. Large areas outside tests/ci (examples, most LLM live tests, the beta agent, and the browser_harness CLI daemon) are not part of that matrix. A separate eval workflow can POST the PR commit to an external eval platform for owner, member, or collaborator PRs.
- pytest is configured in pyproject.toml: timeout 300, asyncio_mode auto, testpaths tests, addopts -svx --strict-markers --tb=short --dist=loadscope, markers slow, integration, unit, and asyncio. bin/test.sh runs uv run pytest --numprocesses auto tests/ci. — [pyproject.toml#L191-L211](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/pyproject.toml#L191-L211) [bin/test.sh#L1-L9](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/bin/test.sh#L1-L9)
- tests/ci is grouped into browser/, infrastructure/, interactions/, models/, security/, plus many top-level test_*.py files covering agent planning, tools, DOM, filesystem, MCP, sandbox structured output, variable substitution, and cloud events. — [tests/ci/conftest.py#L1-L40](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/tests/ci/conftest.py#L1-L40)
- tests/ci/conftest.py autouse fixture forces SKIP_LLM_API_KEY_VERIFICATION, ANONYMIZED_TELEMETRY=false, and placeholder cloud URLs so tests do not talk to production. create_mock_llm builds a BaseChatModel that returns scripted AgentOutput actions. The module docstring says tests must not connect to production services. — [tests/ci/conftest.py#L1-L80](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/tests/ci/conftest.py#L1-L80)
- CLAUDE.md requires real objects in tests, forbids mocking except the LLM, and forbids real remote URLs such as google.com; HTML fixtures should be served with pytest-httpserver. It also says files that pass move into tests/ci and that @pytest.mark.asyncio is unnecessary because asyncio_mode is auto. — [CLAUDE.md](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/CLAUDE.md)
- The GitHub Actions test workflow checks out the repo, caches Playwright Chromium, lists tests/ci test_*.py into a JSON matrix, and runs each file on ubuntu-latest with timeout-minutes 4. It injects OPENAI, ANTHROPIC, GOOGLE, GROQ, AZURE, BROWSER_USE, and OPENROUTER API keys from secrets when present. — [.github/workflows/test.yaml#L27-L120](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/.github/workflows/test.yaml#L27-L120)
- Security-focused tests live in tests/ci/security and include domain filtering, download filename sanitization, IP blocking, MCP allowed domains, security flags, sensitive data, and upload path containment. — [tests/ci/conftest.py](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/tests/ci/conftest.py)
- pyright include is browser_use, examples, and tests, but excludes tests/scripts, tests/old, browser_use/dom/playground, browser_use/llm/oci_raw, browser_use/llm/tests/test_chat_models.py, browser_use/llm/tests/test_single_step.py, and browser_use/beta/service.py. Those paths are not held to the type checker. — [pyproject.toml#L145-L174](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/pyproject.toml#L145-L174)
- Lint CI is three jobs: ruff check --select PLE, pre-commit run --all-files on Python 3.11, and pyright after uv sync --dev --all-extras. Dev dependencies pin pytest==9.0.2, pytest-asyncio==1.3.0, pytest-httpserver==1.1.3, pyright==1.1.408, and ruff==0.14.14. — [.github/workflows/lint.yml#L22-L54](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/.github/workflows/lint.yml#L22-L54) [pyproject.toml#L227-L247](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/pyproject.toml#L227-L247)
- eval-on-pr.yml, for PRs whose author association is OWNER, MEMBER, or COLLABORATOR, POSTs the head SHA to secrets.EVAL_PLATFORM_URL/api/triggerInteractionTasksV6 with a default test case of InteractionTasks_v8. This is an external eval, not the local pytest suite. — [.github/workflows/eval-on-pr.yml#L1-L40](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/.github/workflows/eval-on-pr.yml#L1-L40)
- browser_use/llm/tests contains live-model tests (test_chat_models.py, test_single_step.py, provider cache and schema tests) that are not under tests/ci and are partly excluded from pyright, so they do not run in the per-file CI matrix. — [pyproject.toml#L162-L167](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/pyproject.toml#L162-L167)
- There is no tests/ci coverage of the browser_harness daemon itself; CLI tests that exist (tests/ci/test_browser_use_cli.py is in the tree) can only cover this repo's dispatcher. The harness package is an external dependency (browser-harness==0.1.13). — [pyproject.toml#L49-L50](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/pyproject.toml#L49-L50) [browser_use/cli.py#L185-L202](https://github.com/browser-use/browser-use/blob/d8110c5ff87ccba887aaa726cdb780f2f84bef8d/browser_use/cli.py#L185-L202)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)