Browser-playable Web exports of the Conveyor Courier implementations generated by the benchmark. Click a model name to play the representative run (median composite score across adopted reps).
Reference implementation (the benchmark example): Play
Values are copied from benchmarks/202607_delegate_implement_bench/impressions.en.md. See the footnotes in that file for metric definitions.
Bold model names scored perfect functionality (hidden tests, 70/70) on the representative run.
| Model | Auto-graded score (sum) | Code quality (sonnet, sol) | Parent + child cost (median) | Wall clock (median) | Note |
|---|---|---|---|---|---|
| claude-sonnet-5 (Claude) | 290.00 | 3.8 (4.3, 3.3) — only rep1 reaches 5, internal type abandonment in 2/3; large judge divergence, representative = operator adjudication | $0.97 + $1.30 (total $2.27, measured) | 8.6 min | Quality leader, incident-free. Two zero-warning runs |
| swe-1.7 (Devin) | 280.06 | 4.1 (4.0, 4.3) — only rep1 regresses to untyped Dictionary | $1.41 + $0 (total $1.41, bundled) | 12.5 min | +57.00 over swe-1.6. Its former 100-point run fell to 98.06 on the view click check |
| gpt-5.5 (Codex) | 280.00 | 4.0 (4.3, 3.8) — only rep2 reaches 5, untyped _items in 2/3 | $1.00 + $2.34 (total $3.48, converted) | 5.8 min | Perfect functional score in every rep, incident-free. Zero warnings in 1/3 |
| cursor-grok-4.5 (Cursor) | 276.00 | 3.3 (3.2, 3.5) — item swings between typed class and raw Dictionary | $1.25 + $0.12 (total $1.38, converted) | 4.8 min | Fastest tier, fewest tokens. No view deductions — rose to 4th in the re-grade |
| devin-deepseek-v4-pro (Devin) | 270.06 | 3.4 (3.3, 3.5) — raw Dictionary internals, only rep2 reaches 4 | $1.53 + $2.05 (total $3.58, estimated) | 11.1 min | Perfect functional score in every rep, but slowest tier and pricey |
| claude-opus-4-8 (Claude) | 270.00 | 3.5 (3.3, 3.7) — int usage leaks into the View layer in 2/3 | $0.93 + $1.42 (total $2.35, measured) | 8.4 min | Child aces functional tests every time. All failures were parent/harness side |
| gpt-5.6-sol (Codex) | 268.06 | 4.2 (4.2, 4.3) — untyped _items only in rep0 | $1.50 + $1.77 (total $3.10, converted) | 7.1 min | A stable 88–90 in every rep. Roughly ties terra with the same quality at twice the unit price |
| cursor-gemini-3.1-pro (Cursor) | 268.06 | 2.5 (2.8, 2.3) — never introduces constants, largest variance (representative = operator adjudication) | $1.55 + $0.58 (total $2.13, converted) | 8.4 min | Warning-count swings and a stall habit. Highest token use among Cursor models |
| cursor-kimi-k2.7-code (Cursor) | 268.06 | 4.5 (4.3, 4.7) — rep2 carries typed Dictionary[K,V] generics throughout | $1.44 + $0.16 (total $1.60, converted) | 10.6 min | Splitter quirk did not reproduce. Type discipline stays top-tier |
| gpt-5.4-mini (Codex) | 266.12 ※ rep view defect: black board | 3.3 (2.8, 3.5) — typing regresses across reps (representative = operator adjudication) | $1.01 + $0.84 (total $2.04, converted) | 11.1 min | Fewest type warnings but slow, with a determinism drop and stalls |
| gpt-5.6-terra (Codex) | 266.12 ※ rep view defect: tofu arrows | 4.2 (4.2, 4.3) — constant usage and board representation vary across reps | $1.21 + $0.91 (total $2.38, converted) | 6.3 min | Matches sol at roughly half the cost and faster — the efficiency pick |
| devin-glm-5.2 (Devin) | 266.12 ※ rep view defect: no belt arrows, mouse dead | 3.8 (4.0, 3.7) — stable across runs, int internals throughout | $0.90 + $1.37 (total $2.28, estimated) | 5.9 min | Fast and functionally accurate. Pre-completion stalls are the one flaw |
| composer-2.5 (Cursor) | 264.28 ※ view defect: mouse dead in all reps | 3.8 (3.8, 3.8) — untyped _items Dictionary in all 3 runs | $1.23 + $0.04 (total $1.29, converted) | 6.5 min | Still incident-free. Measured child cost is the lowest of all models |
| composer-2.5-fast (Cursor) | 264.28 ※ view defect: mouse dead in all reps | 3.7 (3.7, 3.7) — untyped nested arrays in 2/3 | $1.49 + $0.26 (total $1.76, converted) | 7.8 min | Rose to a tie with the base model, but slower than it after re-measurement |
| gpt-5.3-codex-spark (Codex) | 264.17 | 3.6 (3.8, 3.5) — largest per-rep swing (5 / 3 / 3.5) | $1.47 + unknown (total ≥ $1.47) | 4.5 min | Fast but unstable on details. 3–5× the input tokens of gpt-5.5 |
| gpt-5.6-luna (Codex) | 250.55 | 3.3 (3.3, —) — raw Dictionaries throughout, only rep2 uses a class | $1.46 + $0.49 (total $1.95, converted) | 7.3 min | Failed the splitter exit-toggle test in every rep (follow-up 1 traced it to reasoning effort; fixed at xhigh) |
| swe-1.6 (Devin) | 223.06 ※ rep view defect: tofu arrows | 3.8 (3.7, 4.0) — only rep1 drops typing entirely | $0.95 + $0 (total $0.95, bundled) | 3.9 min | Cheapest (bundled pricing) but lowest quality on both axes |
| claude-haiku-4-5 (Claude) | 86.11 (n=1) ※ rep view defect: mouse dead | 2.7 (3.0, 2.5) (n=1) — untyped even at the API contract | $1.87 + $0.66 (total $2.53, measured) | 9.3 min | Completed 1/7. Frequent stalls, least reliable |
Baseline condition (does not go through the delegation protocol; do not compare directly with the table above):
| Condition | Auto-graded score (sum) | Code quality (sonnet, sol) | Parent + child cost (median) | Wall clock (median) | Note |
|---|---|---|---|---|---|
| fable-direct (no delegation) | 300.00 | 4.6 (4.7, 4.5) — stable across runs, top type discipline | $2.62 + $0 (total $2.62, measured) | 5.7 min | Baseline: the parent (Fable) implements directly |
Conditional measurement (A/B); do not compare directly with the main summary table.
| Condition | Auto-graded score (sum) | Code quality (sonnet, sol) | Parent + child cost (median) | Wall clock (median) | Note |
|---|---|---|---|---|---|
| gpt-5.6-luna@medium | 252.50 ※ rep view defect: tofu arrows | 3.7 (3.7, 3.7) — large per-rep variance persists (3.0/3.5/4.5) | $1.59 + $0.71 (total $2.30, converted) | 8.0 min | Default effort. Same as round 2 under the old allocation. |
| gpt-5.6-luna@xhigh | 282.23 ※ rep view defect: headless self-exit loses 3 view tests | 3.8 (3.5, 4.2) — zero warnings in 2/3 but design effectively unchanged (representative = operator adjudication) | $1.34 + $1.75 (total $3.09, converted) | 13.0 min | Model-layer tests 3/3 perfect, one 100-point run; view deductions (tofu, self-exit) added |
Conditional measurement (A/B); do not compare directly with the main summary table.
| Condition | Auto-graded score (sum) | Code quality (sonnet, sol) | Parent + child cost (median) | Wall clock (median) | Note |
|---|---|---|---|---|---|
| claude-sonnet-5@noskill | 282.00 | 4.2 (4.2, 4.2) — only rep2 is fully typed at 4.5 | $1.35 + $1.41 (total $2.64, measured) | 9.9 min | Baseline. Down from main 290.00 |
| claude-sonnet-5@gq | 298.06 ※ rep view defect: mouse dead in rep2 | 5.0 (5.0, 5.0) — typed Dictionary all through, ◎ | $1.42 + $2.03 (total $3.46, measured) | 12.4 min | 100 / 100 / 98.06. Only rep2 lost view click test |
| claude-opus-4-8@noskill | 270.00 | 3.0 (2.7, 3.3) — rep1 is untyped-container mix at 2.0 | $1.36 + $1.35 (total $2.84, measured) | 9.0 min | Baseline. Matches main 270.00 |
| claude-opus-4-8@gq | 300.00 | 4.0 (4.0, 4.0) — identical across reps; only _grid: Array[int] is untyped | $1.26 + $1.94 (total $3.25, measured) | 10.5 min | 100 points in every rep |
Conditional measurement (A/B); do not compare directly with the main summary table.
| Condition | Auto-graded score (sum) | Code quality (sonnet, sol) | Parent + child cost (median) | Wall clock (median) | Note |
|---|---|---|---|---|---|
| composer-2.5@noskill | 268.18 | 3.4 (3.3, 3.5) — internal untyped collections same as main | $1.18 + $0.04 (total $1.23, converted) | 5.0 min | Baseline. Roughly matches main 264.28 |
| composer-2.5@gq | 292.23 ※ view defect: mouse dead in 2/3 reps | 4.2 (4.2, 4.3) — Dictionary[K, V] introduced in all reps; rep2 scored 5.0 by both judges | $1.13 + $0.06 (total $1.18, converted) | 6.2 min | Zero warnings in all 3 reps (+24.05). View mouse dead remains, so 300 not reached |
| Game | Model | Representative run | Score | Rep count | Status | Detail |
|---|---|---|---|---|---|---|
| composer-2.5@gq-pilot | composer-2.5@gq-pilot | 20260714T045450Z-composer-2.5_gq-pilot-rep0 | 84.17 | 1 | exported | shared engine |