Any runner via JUnit XML
No native reporter? Any runner that writes JUnit XML — pytest, Go, Ruby, JUnit, PHPUnit, and most others — maps onto the same conventions through the CLI. A JUnit upload yields the same intelligence as the native reporters: the cases are fed through the same recorder, so identity, flaky scoring, and history all work identically.
Upload in two steps
Write the results file with your runner, then upload it:
pytest --junitxml=junit.xml
npx flakemetry junit junit.xmlThe upload reads FLAKEMETRY_ENDPOINT and FLAKEMETRY_TOKEN from the environment, or you can pass --endpoint and --token explicitly.
Examples by runner
# pytest
pytest --junitxml=junit.xml && npx flakemetry junit junit.xml
# Go (with gotestsum)
gotestsum --junitfile junit.xml && npx flakemetry junit junit.xml
# Ruby (RSpec + rspec_junit_formatter)
rspec --format RspecJunitFormatter --out junit.xml && npx flakemetry junit junit.xml
# PHPUnit
phpunit --log-junit junit.xml && npx flakemetry junit junit.xmlHow cases are mapped
Each <testcase> becomes a test execution:
- suite comes from
classname. - file path comes from the
fileattribute, or is derived from a dottedclassnamewhenfileis absent. - status is
failfor<failure>or<error>,skipfor<skipped>, otherwisepass. - duration is
time(seconds) converted to milliseconds. - error carries the failure message, type, and text as the stack.
Pass --fail-on-error if you want the upload step itself to exit non-zero when delivery fails; by default it never breaks your build.
Allure results
Already writing Allure results? Upload the directory. Allure keeps every attempt of a retried test and JUnit does not, so this also gives you the retry evidence:
npx flakemetry allure allure-resultsA test that failed and then passed on retry is recorded as flaky, and failed and broken results both count as failures. A test Allure reports as skipped because a hook failed is recorded as skipped, since it never ran.
XCUITest
An .xcresult bundle is large and binary. Xcode can write a JSON summary, which keeps every repetition from -retry-tests-on-failure:
xcrun xcresulttool get test-results tests --path Test.xcresult > xcuitest-tests.json
npx flakemetry xcresult xcuitest-tests.jsonThe summary holds no start times, so the run is dated when it is uploaded. Upload it from the job that ran the tests.
FLAKEMETRY_VARIANT works with both, the same way as with flakemetry junit.
Uploading without the CLI
If your CI cannot run Node at all, post the report straight to the API — the server parses it into exactly the same run batch the CLI would have sent:
curl -X POST "$FLAKEMETRY_ENDPOINT/v1/ingest/junit" \
-H "Authorization: Bearer $FLAKEMETRY_TOKEN" \
-H 'Content-Type: application/json' \
-d "$(jq -n --arg xml "$(cat junit.xml)" '{
idempotencyKey: "build-\($ENV.CI_RUN_ID)",
resource: { ciProvider: "other", commitSha: $ENV.COMMIT_SHA, branch: $ENV.BRANCH, trigger: "push" },
xml: $xml
}')"The idempotencyKey makes the upload safe to retry: replaying the same report under the same key is deduplicated rather than counted twice. A different report under a repeated key is stored as its own run. See the API reference for the full request schema.
Python? Prefer the native plugin
pytest-flakemetry records more than the JUnit round-trip can carry: parameter values travel as structured params instead of being baked into the test name, and retries are recognised, so a test that passes on rerun is reported as flaky. See Reporters.