Evaluations

An evaluation tracks the analysis of one video. Use create() to wait for the result, or submit() to start a job and retrieve() to check it later. The SDK examples show both flows with complete code.

Read the result

An evaluation settles at completed, partial, or failed.

Status	Meaning
`queued` / `processing`	Still running. Retrieve the same ID later.
`completed`	All requested checks finished. Read `result`.
`partial`	Some checks finished. Read `result` and `error` for the missing checks.
`failed`	No result is available. Read `error.code` and `error.message`.

result.summary describes the overall assessment. result.glitches contains the findings; an empty list means no findings were reported by the completed checks.

Finding type	Details
`visual_glitch`	Description and `region`, including the affected frames and bounding boxes.
`prompt_misalignment`	Description, `prompt_segment`, and `severity` from 1 (minor mismatch) to 5 (requirement absent).

Only read a finding’s type-specific fields after checking its type. See error codes for failed or partial evaluations.

Evaluation history

iterate() fetches pages automatically, newest first. Use status to filter results. The examples below use an initialized client.

Python
TypeScript

for evaluation in client.evaluations.iterate(status=["completed", "partial"]):
    print(evaluation.id)

for await (const evaluation of client.evaluations.iterate({
  status: ["completed", "partial"],
})) {
  console.log(evaluation.id);
}

For multiple videos, submit one evaluation per video, store each ID, and pace new submissions to your account’s limits.

Get started

Models

Guides

Read the result

Evaluation history

​Read the result

​Evaluation history

Read the result

Evaluation history