gotestyourself/gotestsum
github.com/gotestyourself/gotestsum ↗ · 164 files · 766 KB
gotestsum gotestsum runs tests using go test -json, prints formatted test output, and a summary of the test run.
Overview
gotestsum is a Go test runner and reporting CLI for local development and CI. It wraps go test -json, turns test events into configurable terminal output and summaries, and exports JSON or JUnit reports. Its distinctive capabilities include targeted failure reruns, file watching with debugger shortcuts, slow-test analysis and source rewriting, and timing-based CI matrix generation.
System map
The executable is a thin router over the main test-running command and two analysis tools. Test execution crosses a subprocess boundary, while parsing and reporting operate on an in-memory Execution model. Integration centers on terminal output, report files, executable hooks, and filesystem notifications. Build and release definitions describe a portable CLI binary rather than a hosted service.
- main calls route(os.Args), preserves errors implementing ExitCode(), and exits with status 3 for other command errors.
- toolRun dispatches slowest and ci-matrix explicitly; help and ? are handled by both the root router and tool router.
- cmd.Run owns flag parsing, logging setup, build-version reporting, watch dispatch, and ordinary execution.
- The module declares Go 1.24.0 and dependencies for flags, colors, filesystem notifications, shell-like argument splitting, concurrent scanning, and Go package loading.
- testjson exposes ScanTestOutput, ScanConfig, and EventHandler; Execution owns package maps and a separately locked error list.
- cmd's event handler bridges parsed events to terminal formatters and JSON files; final JUnit generation is delegated to internal/junitxml.
- ./do provides binary, binary-static, lint, and golden-update tasks; GitHub Actions builds the executable and uses it to run the repository with race detection.
- GoReleaser builds CGO-disabled binaries for five operating systems and multiple architectures, embeds cmd.version, and defines checksums; CircleCI publishes on matching version tags.
Feature catalog
The primary feature is configurable presentation of Go test events with a final failure-oriented summary. Automation features export reports, retry failed tests, limit failures, and invoke external commands. Local workflows include file watching, debugger integration, and desktop notification examples. Two tool commands consume historical event data for slow-test analysis and CI partition planning.
- NewEventFormatter supports package, test-name, testdox, dots, standard quiet/verbose/JSON, GitHub Actions, debug, and none formats, including compatibility aliases.
- --format-icons selects default, hivis, text, codicons, octicons, or emoticons; --format-hide-empty-pkg applies to supported compact formats, and --no-color controls global color output.
- --hide-summary selectively hides skipped, failed, errors, or output sections; PrintSummary still prints the final test-count and elapsed-time line.
- --jsonfile records raw events, --jsonfile-timing-events records terminal events, and --junitfile supports project naming, name/classname formatting, and empty/skipped filtering.
- --rerun-fails defaults to two reruns when supplied without a value, limits initial failures to ten by default, supports root-test reruns and a report file, and optionally aborts on data races; --max-fails stops scanning at a failure threshold.
- --raw-command runs a supplied executable instead of prepending go test; documented uses include stdin replay and compiled test binaries through test2json, with GOVERSION available when Go is absent.
- --watch reruns changed packages, supports screen clearing and per-package working directories, and offers Unix r/d/a/l/u shortcuts; post-run hooks can invoke notification examples backed by platform-specific executables.
- tool slowest prints median-based slow-test timings or inserts --skip-stmt into source; tool ci-matrix reads package names from stdin and timing-file globs to emit a GitHub Actions matrix.
Data flow
Arguments and environment defaults determine the executable, package selectors, formatter, and report destinations. Subprocess stdout supplies JSON events while stderr supplies displayed diagnostics and selected execution errors. Execution accumulates test state before handlers render or persist events. Reports and analysis tools reuse this event model, but raw JSON persistence and synthesized state have different contents.
- setupFlags disables interspersed parsing; cmd.Run stores remaining arguments for the child, and package selection prioritizes rerun package, --packages, TEST_DIRECTORY, then a supplied default.
- startGoTest connects inherited stdin, captures stdout/stderr pipes, applies an optional child working directory, and starts the executable using exec.CommandContext.
- ScanTestOutput reads stdout and stderr in two errgroup goroutines; readStdout attaches RunID and updates Execution before invoking Handler.Event.
- Execution routes build-output into errors, package events into Package.addEvent, and named test events into Package.addTestEvent; test instances receive package-local IDs, RunID, timestamps, and attributes.
- Output is indexed by test ID with package output at ID 0; successful root tests release output, while subtest and skipped-output retention supports attribution workarounds.
- Err receives stderr before filtering; module-download messages, GODEBUG cache messages, and warning-prefixed lines are excluded from Execution.Errors, while other lines are retained.
- eventHandler writes original JSON bytes before formatting, excludes artificial events with empty bytes, syncs logs before finishRun, and supplies report paths plus execution counts to postRunHook.
- ci-matrix scans each matched report into package elapsed times, selects the nearest-rank 85th percentile, greedily assigns packages to the least-loaded bucket, and serializes include entries with estimatedRuntime and package strings.
Behavior traces
Source inspection followed startup and ordinary execution, targeted reruns, report persistence, watch-triggered execution, and both analysis tools. The normal path separates stream scanning from process waiting and final reporting. Reruns preserve historical Execution state while tracking only current retry failures for continuation. Watch mode follows its own execution path and therefore has different option behavior.
- Startup follows main → route → cmd.Run → setupFlags → Parse → setupLogging; version requests return early, watch requests call runWatcher, and other requests call run.
- Ordinary execution follows run → Validate → goTestCmdArgs → startGoTestFn → newEventHandler → ScanTestOutput → Flush → Wait; without arguments it constructs go test -json with ./... unless package selection overrides it.
- Malformed stdout normally causes readStdout to fail and stopOnError to cancel the child; ignore mode forwards malformed lines to Err, while FAIL-prefixed bad events are handled by a separate continue branch.
- After a failed initial process, run checks hasErrors and the initial failure limit, then rerunFailed filters failures, builds escaped anchored -test.run expressions, starts each retry, scans into shared Execution with a new RunID, and records remaining failures.
- Persistence completes through handler.Flush → finishRun → PrintSummary → writeJUnitFile → junitxml.Write → postRunHook; JUnit or hook errors replace the pending process error, otherwise finishRun returns it.
- Watch execution follows runWatcher → filewatcher.Watch → loadPaths → handleEvent → runTests → watchRuns.run → runSingle; changed .go paths trigger package runs, and watch-chdir sets the child directory and selector ./.
- Slow-test analysis follows slowest.Run → run → jsonfileReader → ScanTestOutput → aggregate.Slowest; --skip-stmt adds parseSkipStatement → writeTestSkip → packages.Load → rewriteAST → writeFile instead of printing.
- Matrix generation follows matrix.Run → run → readPackages → readTimingReports → packageTiming → packagePercentile → bucketPackages → writeMatrix; fewer than two partitions or an empty timing pattern fails before reading inputs.
Testing signals
Tests and CI were inspected, not executed; no passing result is asserted. The inspected suite uses Go testing with gotest.tools assertions, golden files, filesystem fixtures, subprocess helpers, and go-cmp. It combines event-log regressions with selected real subprocess and filesystem integration tests. Coverage observations are bounded to the files examined, and platform exclusions or short-mode skips materially limit some paths.
- cmd's integration suite uses testing, gotest.tools/v3 assert/env/fs/golden/icmd/poll/skip, while event-state tests additionally use go-cmp; these declarations and assertions were inspected only.
- execution_test.go exercises minimal scanning, Stop invocation on handler error, raw-event parsing, synthesized missing failures, non-JSON tolerance, and GODEBUG filtering.
- Formatter tests replay stored stdout/stderr through fakeHandler and compare golden output; fixtures are documented as generated from tagged test packages, with coverage, shuffle, panic, and timeout variants.
- cmd tests cover failure-limit rejection, build-error and panic rerun guards, stdin replay, pre-hook JSON synchronization, and timing-only JSON output; the synchronization test skips Windows.
- Rerun integration cases exercise eventual success, continued failure, and build-error aborts; short mode skips them, and the interrupt test skips Windows while asserting exit status 130.
- JUnit tests use deterministic timestamps and GOVERSION for golden comparisons of ordinary output, empty-package filtering, skipped-test filtering, and test attributes.
- Unix Watch integration checks filesystem-triggered events and a/r/d/u controls without running actual test subprocesses; matrix tests cover percentile/bucketing and serialized output, while the inspected slowest tests cover help and the preset statement rather than end-to-end rewriting.
- Inspected GitHub Actions runs build plus race-enabled tests across stable/oldstable Go on Linux, Windows, and macOS; CircleCI defines lint and release builds. Their configuration is evidence of intended checks, not executed CI success.
Risk map
Security-sensitive boundaries are primarily local executable invocation and user-selected filesystem paths. Source observations identify report truncation, scanner limits, mutable reporting state, and differences between ordinary and watch execution. Release tooling also downloads external programs or scripts. These are source-level observations and explicitly labeled hypotheses, not confirmed vulnerabilities or runtime reproductions.
- Raw commands, hooks, and Delve execute user-selected or PATH-resolved programs with local process privileges; hooks inherit os.Environ, and commandValue uses shlex.Split without automatically invoking a shell.
- JSON, timing, JUnit, and rerun-report paths are passed to os.Create, which truncates existing files; the inspected writers do not constrain destinations to the repository root.
- --skip-stmt rewrites package-loader source paths directly with os.Create, promotes slow subtests to root functions, and does not add imports; earlier file writes can precede a final missing-test error.
- Both stream readers use bufio.NewScanner without increasing its token buffer; sufficiently large event or diagnostic lines can therefore terminate scanning with the wrapped scanner error.
- JUnit HideSkippedTests mutates each Execution-owned Package by decrementing Total and clearing Skipped; because the hook runs afterward, its TESTS_TOTAL and TESTS_SKIPPED can reflect filtered state rather than the summary printed earlier.
- Hypothesis: handler initialization errors deserve process/resource-cleanup review because run starts the child before creating the handler, returns before Wait on initialization failure, and defers handler.Close only after successful creation. (inferred)
- Watch runSingle explicitly lacks rerun-fails support and omits IgnoreNonJSONOutputLines and the ordinary signal-derived exit-code branch; Unix Watch tests exclude Windows, whose terminal adapter disables shortcuts.
- CircleCI verifies the downloaded GoReleaser archive checksum but pipes a remote lint installer into sh; the development Dockerfile also uses a remote installer and defaults to Go 1.18 despite go.mod requiring 1.24.0.
Extension points
Extension mechanisms are explicit Go interfaces, factories, and command switches. The reusable event parser allows callers to provide readers, callbacks, cancellation, and existing execution state. Reporting and automation extensions fit through formatters, JUnit configuration, and post-run executables. New commands require direct router and help integration.
- Add a tool command through main.toolRun's dispatch switch and usage text, following the existing Run(name, args) interface used by slowest and matrix.
- New terminal formats plug into EventFormatter.Format and NewEventFormatter's explicit factory switch; unknown names return nil and become a command error.
- EventHandler.Event and Err support custom consumers, with an Event error stopping scanning; ScanConfig accepts custom readers, Stop callbacks, RunID, and an existing Execution.
- A caller can omit Handler and Stderr for standalone log parsing; ScanTestOutput supplies a no-op handler and empty stderr reader.
- JUnit Config accepts FormatFunc callbacks for suite names and testcase classnames, with identity defaults; CLI full/relative/short options adapt these callbacks.
- --post-run-command integrates external automation through argument parsing, output writers, report paths, elapsed time, and TESTS_* counters; notification examples consume those counters.
- filewatcher.Watch accepts a run(Event) callback, separating filesystem/keyboard event detection from command execution; Event carries package path, appended arguments, and debug mode.
- ByElapsed accepts a timing-reduction function, slowest supports custom parsed Go statements, and rerunFailsFilter selects either root cases or FilterFailedUnique; these are concrete policy seams for aggregation, rewriting, and retries.
Contributor handoff
Begin with the executable router, command lifecycle, event model, and handler boundaries before changing behavior. Preserve test-instance identity, compatibility workarounds, and distinctions between raw event logs and synthesized failures. Tests and CI were inspected, not executed; future execution requires authorization beyond this read-only investigation. Use the bounded evidence here as navigation, then inspect the affected implementation and adjacent tests.
- Read README.md, go.mod, main.go, cmd/main.go, cmd/handler.go, and testjson/execution.go first to establish user contracts, toolchain requirements, routing, execution ordering, and event state.
- Keep flag changes aligned with setupFlags, options.Validate, and child argument construction; reruns with forwarded Go arguments require --packages, and -(test.)failfast is rejected.
- Preserve ID and RunID semantics for repeated tests and reruns; JSON export uses original event bytes, so synthetic EOF failures appear in state and summaries but are deliberately omitted from JSON files.
- Treat EventHandler callbacks as potentially concurrent: ScanTestOutput launches stdout and stderr readers separately, and Execution protects its error list with a mutex.
- Review both run and runSingle when changing lifecycle or option propagation; watch execution is a documented copy without retry support, and Windows keyboard handling differs from Unix.
- For separately authorized validation, target the affected Go package first, then consider CI's build and ./gotestsum -f testname -- ./... -race -count=1 sequence; this investigation inspected those tests and CI definitions without executing them.
- Golden fixtures encode output contracts, and ./do update-golden writes expected results; slowest's --skip-stmt also writes source, so neither belongs in a read-only verification pass.
- Resolve documentation drift from implementation: README lists GOTESTSUM_FORMAT among hook variables, but postRunHook does not explicitly inject it; Docker's default Go version also predates the module directive.
Languages
| Language | Files | Share of source |
|---|---|---|
| Go | 67 | 100% |
| YAML | 5 | docs/config |
| Markdown | 2 | docs/config |
| HTML | 1 | docs/config |
Start reading here
Probable source entrypoint: main.go — inferred from the directory tree and manifest, not the upstream docs.
Source roots
cmd/ (43 files) internal/ (26 files) contrib/ (8 files)
Source layout — cmd/
| Path | Files |
|---|---|
cmd/internal/ | 1 |
cmd/testdata/ | 25 |
cmd/tool/ | 7 |
cmd/flags.go | 1 |
cmd/flags_test.go | 1 |
cmd/handler.go | 1 |
cmd/handler_test.go | 1 |
cmd/main.go | 1 |
cmd/main_e2e_test.go | 1 |
cmd/main_test.go | 1 |
cmd/rerunfails.go | 1 |
cmd/rerunfails_test.go | 1 |
cmd/watch.go | 1 |
Repository structure
| Top-level path | Files |
|---|---|
testjson/ | 72 |
cmd/ | 43 |
internal/ | 26 |
contrib/ | 8 |
.project/ | 4 |
.github/ | 2 |
.circleci/ | 1 |
Docs found: README.md
Package
go.mod — gotest.tools/gotestsum, 0 declared dependencies.
CI & config entrypoints
README.md— Repository readme
Structural health signals
Average score 9.8/10 across 67 analyzed source files; 0 hotspots surfaced.
testjson/execution.go— 917 lines, score 6.9 (Large file, High churn)testjson/format.go— 515 lines, score 7.9 (Large file, Moderate churn)cmd/main.go— 557 lines, score 7.9 (Large file, Moderate churn)
These are deterministic review leads — size, churn, and structural signals — not proof of a bug in this project.
Recent history
3cf14356a16c2026-04-14 — Merge pull request #563 from alexandear-org/refactor/remove-unused-go-versionc4163c3593342026-04-10 — Remove unused Go version checks and related test data06f60b3249912025-09-16 — Merge pull request #528 from alebcay/configurable-version47842dbc0c692025-09-11 — Allow version to be configured at build timec4a0df2e75a22025-09-10 — Merge pull request #526 from gotestyourself/dependabot/go_modules/golang.org/x/sync-0.17.0