API reference

API reference · Account

A shadow test's report, from the dashboard

GET/api/account/shadow/{id}/{action}Signed in session

action is report. A 404 at every path while the operator has not turned shadow testing on.

Parameters

Path and query parameters
FieldTypeWhat it is
id (path)requiredstring
action (path)requiredstringOne of "report", "switch", "revert", "stop", "auto".

Answer 200

The report.

Fields of the 200 answer
FieldTypeWhat it is
idstring
object"shadow.report"
modelstring
statusstring
thresholdnumber
min_samplesinteger
confidencenumber
candidatesarray of object
candidates[].modelstring
candidates[].suggestedboolean
candidates[].replaysobject
candidates[].replays.doneinteger
candidates[].replays.failedinteger
candidates[].replays.pendinginteger
candidates[].replays.skippedinteger
candidates[].agreementobject
candidates[].agreement.methodstring
candidates[].agreement.comparedintegerThe sample count.
candidates[].agreement.agreedinteger
candidates[].agreement.ratenumber or null
candidates[].agreement.intervalobject or nullThe 95% Wilson score interval; null with nothing compared.
candidates[].agreement.interval.lownumber
candidates[].agreement.interval.highnumber
candidates[].agreement.original_ratenumber or nullThe suite method: the original model's pass rate on the suite.
candidates[].readyboolean
candidates[].notified_atinteger or null
candidates[].costobject
candidates[].cost.replay_microsintegerWhat the replays were billed.
candidates[].cost.original_microsintegerWhat the same requests were billed on the original.
candidates[].cost.rationumber or null
candidates[].cost.compare_microsintegerEmbeddings or the judge, billed.
candidates[].latency_msobject
candidates[].latency_ms.p50integer or null
candidates[].latency_ms.p95integer or null
candidates[].original_latency_msobject
candidates[].original_latency_ms.p50integer or null
candidates[].original_latency_ms.p95integer or null
candidates[].failuresarray of object
candidates[].switchedboolean

Errors

  • 401OpenAI shaped error.
  • 404OpenAI shaped error.

Every error answers in one shape, with its type, code and the field it names. The whole list is in the docs' Errors section.

Antimatter