Confirmatory Tasks¶
Confirmatory tasks are the two public task contracts reserved for the most controlled evaluation designs in this release:
| Task | Public experimental question | Interaction |
|---|---|---|
reaction-to-crystallization |
Use reaction assays to adapt a seeded cooling trajectory, then isolate product while balancing yield, purity and particle-size distribution | Campaign |
electrochemical-conversion |
Choose bounded medium and electrolyte conditions, then optimize selective conversion under charge, energy and risk accounting | Campaign |
What “confirmatory” means here¶
It identifies a task role, not a published winner. The task contract, legal operations, public instruments, budgets, termination rules, software qualification and replay boundary are stable enough to support a preregistered comparison. Method selection, training data, resource parity, seeds and statistical claims still belong to the evaluation protocol.
Public evidence¶
Public v0.4 publishes:
- the task cards and typed action/observation contracts;
- deterministic runtime and replay tests;
- reference and generated world-composition qualification;
- sanitized finite evidence and its construction protocols;
- explicit limitations and evidence denominators.
It does not publish Participant rankings, development-only gates, private confirmation materials or v0.5 candidate results. A score produced in the Lab on one seed is an interaction demonstration, not a confirmatory benchmark result.
Minimum comparison contract¶
A formal study should predeclare the frozen release, task contract hash, world/scenario split, seeds, Agent identity, allowed feedback, experiment and method resources, failure handling, primary task-specific outcome and replay verification. Report risk, cost, adaptation and autonomy separately rather than combining unlike quantities into one universal score.
Read Benchmark Design, Evidence & Current Status and the public protocols.