Skip to main content
POST
Create an evaluation

Authorizations

Authorization
string
header
required

Use a key created in the Playground. Keys begin with gk_.

Body

application/json
prompt
string
required

What the video was meant to show. REQUIRED as of 2026-08-27 (previously optional, defaulting to ""); empty or whitespace-only is refused with missing_prompt.

Required string length: 1 - 2500
Pattern: \S
video
object
required

How to give us the video. Exactly one of the three, and the choice is mostly about size. upload_id is the general answer: upload the file first (three calls, one of them straight to storage -- see POST /v1/videos) and reference it here. It is the only option that both reaches the 50 MB file limit and keeps the video private to your account. url is the shortcut when the video is already hosted somewhere we can GET. Note that it has to be publicly reachable; we send no credentials. b64_json sends the bytes inline. Convenient for a small local file, but bounded by the request body limit rather than the file limit -- see POST /v1/evaluations.

model
enum<string>
default:galileo
Available options:
galileo,
gemini
model_version
string

Concrete model version to run. The deployment default applies when omitted.

Minimum string length: 1
glitch_types
enum<string>[]
Minimum array length: 1
Available options:
visual_glitch,
prompt_misalignment
metadata
object

JSON metadata echoed on the evaluation. Serialized size must not exceed 8192 bytes.

Response

Evaluation created.

id
string<uuid>
required
object
string
required
Allowed value: "evaluation"
model
enum<string>
default:galileo
required
Available options:
galileo,
gemini
model_version
string
required
output_schema_version
string
required
created
integer
required

Unix timestamp in seconds.

Required range: x >= 0
status
enum<string>
required
Available options:
queued,
processing,
completed,
partial,
failed
input
object
required
usage
object
required
result
object | null
required
error
object | null

Why the run failed, on a failed one. NULL on a run that did not fail, and ABSENT on rows written before this field existed. Not in required, and that is a correction rather than a preference: it was, and the older rows in the store do not carry the key at all. A client that enforced the contract at runtime — the Python one does, since its models validate — raised on any page deep enough to reach them, while the TypeScript client passed because its types are erased before a response is ever seen. So the contract was not merely wrong, it was wrong in a way that only one of the two clients could report. Test for a VALUE, not for the key.

detectors
object[]

Per-detector state. Older evaluations may omit this field.

video_id
string | null

Stored video identifier when the evaluation used an uploaded video.

attempt
integer

Which try this is. 1 for a run submitted directly; 2 or more for one produced by POST /v1/evaluations/{evaluation_id}/retry. There is a ceiling, so a clip that keeps failing under the same instructions stops being retryable rather than being retried forever.

Required range: x >= 1
timing
object | null

How long this run took. NULL when nothing was measured -- which is every run settled before it was recorded, and is not backfillable. Null and never 0. A run whose latency nobody recorded and a run that took no time are different claims, and only one of them is true. NOT in required, for the same reason error is not: during a rolling deploy some tasks are still the older build, and a response from one of those carries no such key. A client that enforces the contract at runtime -- the Python one does -- would raise on those responses, and it would raise only during a deploy, which is the worst time to be debugging a client. Test for a VALUE, not for the key.

metadata
object | null

Whatever you passed on create, echoed back. NULL when you passed nothing -- the key is always present, its value says whether there was any. Only found by submitting through the API. Every evaluation created in the console carries metadata, so a contract checked against console traffic alone looked correct here.