Answers are checked against the page they came from, builds against the frame they produce. The reward follows what a person would actually see — never a text proxy for it.
We attack every grader before it ships — blank canvases, stub scenes, tampered judges. Anything that lets a non-solution score gets rebuilt.
Domain experts write the rubrics and review the outputs criterion by criterion, each tied to evidence you can point at in the frame or on the page.
Explore datasets and visual examples
Professional questions over long, real documents — reports, handbooks, standards, scanned test records — where the answer sits in a chart, a drawing, a handwritten table or a change between editions. Every answer is graded against the pages it came from.
The agent follows a real illustrated tutorial and delivers the still-editable Blender project, not a picture of one. We reopen it and measure geometry, materials and animation against a reference build the agent never sees.
From a written brief and visual references, the agent models an object or scene in Blender and ships it as a working web experience — exploded assemblies, animated mechanisms, guided tours, first-person walkthroughs.
Browser-native 3D built in code — procedural geometry, materials, lighting, camera controls and interaction — delivered as an editable, runnable Three.js project with multi-view captures.
A WORLD WRITTEN IN WAVESPoseidonExplore the ocean
A complete, playable WebGL / WebGPU game from a bare Vite starter — world, controls, objectives, HUD. Behavior is graded from scripted play sessions, not from the code alone.
These are a few featured datasets — the full catalog covers more task families, with scale and measured difficulty for each. Request access.
The same tasks come in four formats — pick the one that matches how you train.
A full record of each task being solved: every action, every screenshot checked, every fix. It shows a model how the work actually gets done.
Each task in a container with its tools and an automatic grader. The model tries, gets a score and learns from it. Graders are tested against cheating first.
Experts score outputs against a checklist and pick the better of two, noting what's wrong. For training models that judge visual work.
Tasks never used for training, each with a pass rate measured on today's top models — so you can see where yours stands.
This page — datasets, principles, and how outputs are graded.
Representative task packages — briefs, reference files, renders, verifiers and rubrics — on request.
A scoped batch against your runtimes, formats and acceptance criteria, with measured pass rates.
Task inventories, scale and measured difficulty — customer access.
The full catalog — task inventories, runtimes and measured difficulty — is shared under NDA. Request access and we'll respond within one business day.
We'll share representative task packages — reference files, renders, verifiers and rubrics — and scoping for custom programs.