NIMI-research/Tycho

TL;DR: Tycho is an agent harness for ARC-AGI-3 in which a frontier multimodal model, dropped into an unknown 64×64 grid world with no rules or stated goal, accumulates observations and — when it judges it useful — writes an executable world model (a rendered deterministic Moore machine in Python), falsifies it against logged experience, and plans through it. The headline claim is a perfect 100.00 on official competition-mode scorecards across all 25 public games with both GPT-5.6 Sol and Claude Opus 5, and the "actor decides when to formalize" ablation is the interesting scientific bit.

What it is and why it matters

ARC-AGI-3 is the interactive successor to ARC: agents face unfamiliar environments and must discover both the dynamics and the objective by acting. Most harnesses treat this as pure in-context reasoning over screenshots. Tycho's framing — telegraphed by the Tycho Brahe epigraph — is empiricist program induction: keep a durable record of (observation, action, outcome) triples, and convert that evidence into a free-form executable hypothesis with a fixed interface: State, transition, render, outcome. Because the hypothesis is code, it can be checked deterministically against every recorded transition (replay the actions, compare rendered frames), and once it survives falsification it becomes a cheap simulator for planning — no model calls needed to roll out candidate action sequences.

The design deliberately does not force formalization. When writing a world model isn't worth it, the agent reasons directly from typed evidence and notes. That flexibility is one of the four policies being ablated.

The ablation and the numbers

Four policies, identical observation/action/scoring plumbing:

• no_world_model — direct reasoning from evidence.
• single — one actor model writes and uses world_model.py itself.
• orchestrator — actor invokes a dedicated world-model builder when it decides to.
• trigger — the harness invokes the builder when verification falsifies or under-specifies the model.

Two takeaways from the ablation: executable world models help (79 → 88 on the same model), and who decides when to abstract matters — the actor-controlled builder beats the harness-triggered one (88.49 vs 83.07). Letting formalization be a deliberate act of the agent rather than an automatic reflex is the paper's actual claim, and it's a clean, testable one.

Caveats: the README never defines RHAE (presumably a human-relative aggregate ARC-AGI-3 metric), gives no per-game breakdown, no token/cost accounting, and no variance across runs — it explicitly calls the paper configs "long-running, stochastic, and expensive." Two 100.00 scores on 25 public games are impressive but say nothing about the private evaluation set, and public-game mechanics may leak into prompts through iteration. The scorecard IDs and manifests in artifacts/ are the verifiable evidence.

What's in the repo

Fairly complete for a paper artifact: agent, prompts, workspace API, planner and verifier, benchmark runner, Anthropic and OpenAI transports, paper configs, tests, scorecard manifests, and a local replay viewer that shows frame-by-frame evidence, model calls, and world-model diagnostics. Agent-authored Python executes in a fresh network-disabled container (Docker or Finch). Apache-2.0, Python ≥3.12. No paper link in the README yet, and reproducing headline numbers requires frontier-model API keys plus ARC-AGI-3 access — budget accordingly.

Trying it

git clone https://github.com/NIMI-research/Tycho && cd Tycho
python3.12 -m venv .venv && .venv/bin/pip install -e .
make sandbox-image && make sandbox-check   # Docker or Finch required
make validate                              # credential-free test suite
export ANTHROPIC_API_KEY=...               # or OPENAI_API_KEY
tycho-viewer results --host 127.0.0.1 --port 8900

Start with the bounded smoke tests in docs/REPRODUCING.md and browse a replay before touching the paper configs. Even if the 100.00 doesn't survive the private set, the actor-controlled abstraction result and the executable-hypothesis verifier are worth stealing for any interactive-environment agent work.