OpenTelemetry Test Conventions

The canonical span and attribute model every reporter emits to. The machine-readable source of truth is packages/contracts/src/otel.ts (re-exported from @flakemetry/sdk); this document is the human-readable spec. Product rationale lives in ADR-0002 and the wiki.

Conventions version: 0.1.0.

Span hierarchy

test.run                 root span — one CI job / suite invocation
 └─ test.case            one execution of one test (retries are separate cases)
     └─ test.step        a step / hook (added in M2)

M1 emits test.run + test.case. test.step and network/browser child spans arrive in M2.

Resource / run attributes

Key	Example	Meaning
service.name	flakemetry-reporter	standard OTel service identity
flakemetry.project	acme/web	project the results belong to
ci.provider	github_actions	CI provider
ci.run_id	9000001	provider run id — correlates parallel shards into one logical run
test.shard.index	2	shard number (1-based), when the run is sharded across parallel jobs
test.shard.total	4	total shard count, when sharded
vcs.commit_sha	a1b2c3d	commit under test — enables the same-sha flake signal
vcs.branch	main	branch
vcs.pr_number	42	pull request, when applicable
flakemetry.trigger	push	run trigger (push/pull_request/schedule/manual/other)
flakemetry.idempotency_key	gh-9000001-1	one per ingested run; makes re-delivery safe (falls back to the run span trace id). A sharded run sends one key per shard — the reporter appends -shard<index> so parallel shards don't overwrite each other
flakemetry.contract_version	0.1.0	conventions/contract version stamp

Case span attributes

Key	Example	Meaning
test.identity.fingerprint	sha256:…	stable identity (L1), computed by the reporter
test.suite	auth	grouping
test.title	logs in	display name
test.file_path	e2e/auth/login.spec.ts	source location
test.params_hash	9f2c…	parameterized bucket, omitted when absent
test.status	pass | fail | skip | flaky	verdict
test.attempt	2	retry index (1-based)
test.retry_of	0	index of the attempt this one retries, when applicable
test.duration_ms	1834	wall-clock duration
test.artifacts	[{"name":"trace",…}]	JSON array of artifact refs (screenshot/video/trace), workspace-relative paths

Status mapping

Test status	OTel span status
pass, flaky	OK
fail	ERROR (+ exception event carrying type / message / stack)
skip	UNSET

Fingerprint (L1)

sha256(normalized_file_path + ' ' + suite + ' ' + title + ' ' + params_hash) where the path is workspace-relative, POSIX-separated and lowercased. This is the exact-match layer (L1). The server-side identity engine stitches history on top of it: L2 re-links a test whose file moved (same suite + title + params), and L3 re-links a renamed test when — within the same file, suite and params bucket — exactly one prior test has a sufficiently similar title (token-Jaccard ≥ 0.5); ambiguous matches are left as new to avoid collapsing distinct tests. Parameterized cases stay distinct by params_hash.

OTLP → contracts mapping

Reporters export real OTLP spans via @opentelemetry/exporter-trace-otlp-http to POST /v1/traces (OTLP/HTTP JSON). The API normalizes the span tree into a contract-valid ingestRunBatch (@flakemetry/contracts) with otlpToIngestBatch before enqueueing, so downstream stages stay transport-agnostic. Field mapping:

Batch field	Source
resource.*	run/resource attributes above (read from the OTLP Resource, falling back to test.run span attributes)
executions[].{filePath,suite,title,status,attempt,durationMs}	case span attributes
executions[].retryOfIndex	test.retry_of when present, otherwise reconstructed from attempt ordering per fingerprint
executions[].error	the case span exception event (exception.type / exception.message / exception.stacktrace)
run.status	test.run span status (ERROR → failed), or any failing case
idempotencyKey	flakemetry.idempotency_key, else the run span trace id