# Testing signals — flutter/flutter

> Canonical page: https://codevetter.com/unpack/flutter/testing-signals

# Testing signals — flutter/flutter

Scanned commit acb8fd51143a49384799557cb1df6ecead61a55a (2026-09-22). Generated by codevetter codevetter 1.12.0 on 2026-09-22.

Testing is extensive and shard-oriented. dev/bots/test.dart is the single entrypoint: it maps SHARD/SUBSHARD env vars to suite runners under dev/bots/suite_runners and randomizes test order within shards (seeded via --test-randomize-ordering-seed). Framework unit/widget tests live in packages/flutter/test mirroring lib/; tool tests live in packages/flutter_tools/test split into shard dirs (general.shard, commands.shard/{hermetic,permeable}, web.shard, integration.shard, android_*_integration.shard). Engine C++ unit tests sit next to sources (*_unittests.cc). Golden-image testing goes through packages/flutter_goldens (Skia Gold), device-level integration through dev/devicelab and dev/integration_tests, and coverage via .github/workflows/coverage.yml.
- dev/bots/test.dart is the repo-wide test entrypoint: `dart dev/bots/test.dart` runs all tests; SHARD and SUBSHARD env vars select subsets (e.g. SHARD=framework_tests SUBSHARD=widgets, SHARD=tool_tests SUBSHARD=general); --dry-run lists what would run. — [dev/bots/test.dart#L1-42](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/bots/test.dart#L1-L42) [dev/bots/test.dart#L87-140](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/bots/test.dart#L87-L140)
- Tests within a shard are executed in random order to catch inter-test dependencies; --test-randomize-ordering-seed=<n> reproduces a given order. — [dev/bots/test.dart#L35-38](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/bots/test.dart#L35-L38) [dev/bots/test.dart#L103-105](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/bots/test.dart#L103-L105)
- Suite runners are modular: dev/bots/suite_runners/run_*.dart files each own one shard (framework, web, analyze, customer_testing, docs, verify_binaries_codesigned, fuchsia_precache, skp_generator, android hardware smoke, java17 integration, preview integration). — [dev/bots/suite_runners](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/bots/suite_runners) [dev/bots/test.dart#L52-69](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/bots/test.dart#L52-L69)
- Framework tests live in packages/flutter/test mirroring lib/ structure (animation, cupertino, material, widgets, services, semantics, rendering, ...); flutter_test provides testWidgets/pumpWidget and _goldens_io.dart/_goldens_web.dart for golden comparisons. — [packages/flutter/test](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/packages/flutter/test) [packages/flutter_test/lib/flutter_test.dart](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/packages/flutter_test/lib/flutter_test.dart)
- Tool tests are sharded by directory: packages/flutter_tools/test/{general.shard,commands.shard,web.shard,integration.shard,android_java17_integration.shard,android_preview_integration.shard}; commands.shard is further split into hermetic and permeable subdirs. — [packages/flutter_tools/test](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/packages/flutter_tools/test)
- Engine C++ code uses gtest-style unit tests co-located with sources (engine_unittests.cc, animator_unittests.cc, rasterizer_unittests.cc, pipeline_unittests.cc, persistent_cache_unittests.cc) under engine/src/flutter/shell/common; there is also engine/src/flutter/testing for harnesses. — [engine/src/flutter/shell/common/engine_unittests.cc](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/engine/src/flutter/shell/common/engine_unittests.cc) [engine/src/flutter/shell/common/rasterizer_unittests.cc](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/engine/src/flutter/shell/common/rasterizer_unittests.cc) [engine/src/flutter/testing](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/engine/src/flutter/testing)
- Device-lab integration tests run on real hardware via dev/devicelab (bin/run.dart task runner + lib/tasks task library); dev/integration_tests contains ~35 test apps wired into the pub workspace. — [dev/devicelab/bin/run.dart](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/devicelab/bin/run.dart) [dev/devicelab/lib/tasks](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/devicelab/lib/tasks) [pubspec.yaml#L38-67](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/pubspec.yaml#L38-L67)
- CI wiring: .ci.yaml declares per-commit LUCI targets with platform_properties (OS, cores, device_type, kvm, dependencies like android_sdk/open_jdk/gradle_dists versions) and enabled_branches master + release candidates. — [.ci.yaml#L14-80](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/.ci.yaml#L14-L80)
- GitHub Actions runs auxiliary checks only: tool-test-general.yml runs the tool_tests general subshard on PRs touching dev/, packages/flutter_tools/, bin/, engine/, .ci.yaml, DEPS; coverage.yml handles coverage. — [.github/workflows/tool-test-general.yml#L1-35](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/.github/workflows/tool-test-general.yml#L1-L35) [.github/workflows/coverage.yml](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/.github/workflows/coverage.yml)
- Static analysis is enforced: root analysis_options.yaml + analysis_options_common.yaml apply repo-wide lints, and dev/bots/analyze.dart is a custom analysis shard runner (plus check_code_samples.dart, check_examples_cross_imports.dart). — [analysis_options.yaml](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/analysis_options.yaml) [dev/bots/analyze.dart](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/bots/analyze.dart) [dev/bots/check_code_samples.dart](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/bots/check_code_samples.dart)
- Test ownership is tracked in TESTOWNERS at repo root; dev/forbidden_from_release_tests guards that certain tests never run on release branches. — [TESTOWNERS](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/TESTOWNERS) [dev/forbidden_from_release_tests](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/forbidden_from_release_tests)
- Coverage/testing gaps are partially self-reported: framework_coverage is its own shard (run_framework_coverage_tests.dart) implying measured coverage enforcement rather than incidental. (inferred) — [dev/bots/suite_runners/run_framework_coverage_tests.dart](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/bots/suite_runners/run_framework_coverage_tests.dart) [dev/bots/test.dart#L127](https://github.com/flutter/flutter/blob/acb8fd51143a49384799557cb1df6ecead61a55a/dev/bots/test.dart#L127)

Every claim links to its cited files at the pinned commit. Citations identify source evidence; they do not establish runtime correctness. Independent CodeVetter scan — not reviewed by upstream maintainers.

## Public product links

- [CodeVetter](https://codevetter.com/)
- [Download](https://codevetter.com/download)
- [Documentation](https://codevetter.com/docs/)
- [Source](https://github.com/Codevetter/codevetter)