Architecture
Flakemetry is a layered, event-driven platform with one governing constraint: ingestion never blocks CI (ADR-0004). This document is the code-adjacent architecture reference; deeper product context lives in the wiki.
System overview
┌──────────────────────────────────────────────────────────────────────┐
│ INGESTION EDGE │
│ @flakemetry/playwright-reporter · OTLP endpoint · GitHub Action │
└────────────┬───────────────────────────────────────────────────────────┘
│ OTLP/HTTP JSON (ADR-0002), idempotency key per run
▼
┌──────────────────────────────────────────────────────────────────────┐
│ INGESTION API (apps/api) │
│ authenticate → validate against contracts → enqueue → 202 │
└────────────┬───────────────────────────────────────────────────────────┘
▼ durable queue: Postgres FOR UPDATE SKIP LOCKED (ADR-0004)
┌──────────────────────────────────────────────────────────────────────┐
│ WORKERS (apps/worker) │
│ normalize spans → resolve test identity → update flaky score │
│ → cluster failure signature → trigger AI RCA (async, budget-gated) │
└────────────┬───────────────────────────────────────────────────────────┘
▼
┌──────────────────────────────────────────────────────────────────────┐
│ STORAGE │
│ PostgreSQL: relational core + JSONB + rollups (packages/db) │
│ S3/MinIO: artifacts (screenshots, videos, traces) │
└────────────┬───────────────────────────────────────────────────────────┘
▼
┌──────────────────────────────────────────────────────────────────────┐
│ QUERY API (tRPC/REST) → DASHBOARD (apps/web) │
└──────────────────────────────────────────────────────────────────────┘Data flow
- A reporter emits OTel spans per the test semantic conventions and exports OTLP/HTTP with a per-run idempotency key.
apps/apiauthenticates the project token, validates the payload against@flakemetry/contracts, enqueues it and ACKs202immediately.apps/workerdequeues withSKIP LOCKED, materializesrunandtest_executionrows, resolves the stable test identity, incrementally updates the flaky score (ADR-0003) and clusters the failure signature.- New failure signatures trigger asynchronous, budget-gated AI root-cause analysis; known signatures reuse the cached report at zero cost.
- The dashboard reads through the typed query API, hitting rollups rather than raw executions for trends.
Workspace map
The map of every app and package, with what each one does and which ones are published, is kept in one place: the development guide. A test fails when it misses a workspace or marks one published that is not.
Dependency direction is strictly downward: apps depend on packages, packages depend on contracts, nothing depends on apps.
Load-bearing decisions
| Decision | Record |
|---|---|
| Monorepo: pnpm workspaces + Turborepo, contracts as shared source of truth | ADR-0001 |
| OTel-native ingestion, tests as traces | ADR-0002 |
| Explainable statistical flaky scoring, no black-box ML | ADR-0003 |
| Async ingestion: 202 + durable Postgres queue | ADR-0004 |
| Executions and spans stay in Postgres; no columnar store | ADR-0005 |
New load-bearing decisions require a new ADR from the template; superseded decisions are marked, never deleted.
Cross-cutting rules
- Multi-tenancy from day one:
org_idandproject_idon every table, indexed, even in single-tenant self-host. - Storage seams: the queue sits behind an interface so a real broker can replace it at scale without touching producers or consumers. Executions and spans stay in Postgres; ADR-0005 records the benchmark behind that.
- Fail open at the edge: reporters and the GitHub Action never fail a CI job because Flakemetry is unreachable.
- Provider-agnostic AI: hosted or local models are a config change, never a code change.