Your first insight
Once a few runs have landed, the dashboard turns them into answers. Here is the path from raw runs to a decision.
1. The flaky board
The flaky board ranks tests that have needed a retry or failed outside a run where most of the suite failed, by flaky score, worst first. The score is a Beta-Binomial estimate with exponential time-decay — recent behaviour counts for more than last month's. A test with neither is not ranked, because its score only says how little history it has. A single red run still does not brand a test as flaky: quarantine needs enough runs and a score above the bar.
2. Reason codes
Open a test and the score breaks into reason codes — the human-readable evidence behind the number:
- same commit, different result — the strongest flaky signal: the code did not change but the outcome did.
- pass-on-rerun — failed, then passed on retry with no code change.
- intermittent across branches — fails on some branches and not others without a code reason.
Because every score ships with its reasons, you can trust it — or argue with it — instead of taking a black-box number on faith. See Flaky scoring for the model.
3. Test history that survives refactors
A test's history does not reset when you move or rename its file. The test identity engine stitches executions across refactors, so the flaky score reflects the test's real long-run behaviour rather than the age of its current file path.
4. Root-cause on failures
For genuinely new failure signatures, the AI RCA panel offers a structured "likely cause + suggested action". Failures are normalized and clustered first, so only new signatures reach the model — root-cause without a per-failure LLM bill.
5. Guard your pull requests
Wire the GitHub Action in and every PR gets a sticky comment plus a quality gate that distinguishes new failures this change introduced from tests that already flake on the base branch — blocking only what the change actually broke.
6. Track the trend
The Test health page answers the executive question: is flakiness improving or degrading? It reports the tests that passed on a retry each week, the tests that crossed the quarantine bar and came back under it, how long they stayed above it, the current quarantine backlog, and the overall reliability trend — over a time range up to a quarter. More tests passing on a retry week after week is the early warning that flakiness is winning.