Antimatter

POST/v1/evals/{id}/runsAPI key

Chat scope. Every case on every model (at most 12), cheapest model first, each a real dispatch billed to the account as the key that started it (the session's runs are billed to the personal account), with fallback, presets, aliases and the cache off. Each case holds its worst case against budget_micros before it runs, so the run never spends past it; a case that does not fit is skipped. One run of a suite at a time (409). Work starts after the answer; read the run to follow it.

Parameters

Path and query parameters
Field	Type	What it is
id (path)required	string	The suite id (ev_...).

Request body

application/json, required

Request body fields
Field	Type	What it is
models	array of string
max_price_per_1m	number
limit	integer
budget_microsrequired	integer	The most the run may spend, at most 50000000. Each case holds its worst case against it before it runs and is skipped when that does not fit.

Example

{
  "models": [
    "string"
  ],
  "max_price_per_1m": 0,
  "limit": 0,
  "budget_micros": 0
}

Answer 201

The run, queued.

Fields of the 201 answer
Field	Type	What it is
id	string
object	"eval.run"
suite_id	string
status	string	One of "queued", "running", "completed", "budget_exhausted", "cancelled".
trigger	string	One of "manual", "scheduled".
revision	integer
models	array of string
case_count	integer
budget_micros	integer
spent_micros	integer
reserved_micros	integer	Worst cases held for cases in flight.
progress	object
progress.done	integer
progress.total	integer
created_at	integer
started_at	integer or null
finished_at	integer or null
threshold	number
cheapest_passing	string or null	The cheapest model graded on every case at or over the threshold.
summary	array of object
summary[].model	string
summary[].pass_rate	number or null	Weighted share of graded cases passed.
summary[].passed	integer
summary[].failed	integer
summary[].errors	integer
summary[].skipped	integer
summary[].pending	integer
summary[].graded	integer
summary[].cases	integer
summary[].complete	boolean
summary[].cost_micros	integer
summary[].cost_per_case_micros	integer or null
summary[].median_latency_ms	integer or null
summary[].on_frontier	boolean	No other model is as cheap or cheaper and passes as much or more.
failures	array of object	At most 200, each with the model's output clipped to 4000 characters.
failure_count	integer

Errors

400OpenAI shaped error.
401OpenAI shaped error.
403OpenAI shaped error.
404OpenAI shaped error.
409OpenAI shaped error.
503OpenAI shaped error.

Every error answers in one shape, with its type, code and the field it names. The whole list is in the docs' Errors section.

Try it

The same call with curl

curl -X POST https://useantimatter.com/v1/evals/{id}/runs \
  -H "Authorization: Bearer $ANTIMATTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"models":["string"],"max_price_per_1m":0,"limit":0,"budget_micros":0}'