ARC Prize 2026: a language-model agent, and a paper on what to carry to the next level
Carson Rodrigues / November 21, 2026
7 min read • ––– views
ARC-AGI-3 is a set of small puzzle games. An AI agent has to work out the rules by playing, then win, and it's scored on how few moves it needs compared with a person.
I worked on two linked pieces for ARC Prize 2026: a language-model agent that plays the games on Kaggle, and a paper asking what an agent should carry from one level to the next, its plan or the rules it learned. On 21 November, Kaggle's public leaderboard showed the agent at rank 995 of 4,926 teams, with a score of 29.93.
| Competition | ARC Prize 2026: ARC-AGI-3 on Kaggle, and the Paper Track |
| Organiser | ARC Prize Foundation |
| When | Entries closed 26 October, final deadline 2 November 2026. Winners announced 4 December |
| Field | 4,926 teams on the public leaderboard on 21 November |
| Result and rank | Public leaderboard 995 of 4,926 on 21 November 2026; final results not released yet. Paper results due 4 December |
| Built in | About three weeks, from 12 October to 2 November |
| Team | Solo, with Claude Code writing most of the code under my direction |
| Money spent | About $0.20 in API calls for one set of paper experiments. The agent ran on Kaggle's free weekly GPU time. Total not tracked beyond that |
| Code | github.com/rodriguescarson/arc-agi-3 and github.com/rodriguescarson/arc-prize-2026 |
What I built
The agent is a language model that plays each game by writing Python. It looks at the board, tests what its moves do, and then acts. The model is Qwen3.8-Flash-Next, squeezed down (quantised) so it fits on one RTX Pro 6000 card with no internet. I built on Tufa Labs' open Duck code, the wrapper that lets the model run code, as extended by Daniel Franzen, whose open notebook led the earlier milestone.
My part was the plumbing. I wrote the pipeline that builds each variant, runs it on Kaggle's hardware, and sends one scored run per day, which is the daily limit. I added switches to the wrapper, such as a helper that shows the model the moves that won the previous level. Across five variants I got thirteen valid draws, a draw being one scored run on the 55 hidden games. They average 26.35 with a spread (standard deviation) of 3.18. The best, 29.93, came from the plain copy of Franzen's setup with my extras switched off.
The paper's agent is different. It's a small program with no language model that runs on a laptop CPU. It learns what each action does to the objects on screen, plans a route through that learned model, and carries the model and a guessed goal into the next level.
Research
Primary research. The paper's question is simple. When you've solved level k, what helps on level k+1? One answer is the plan, the winning moves. The other is the mechanics, what each action did.
I tested both on 340 recorded human sessions. Replaying a winning plan on the next level cleared it in 0 of 1,186 testable pairs, across 23 games. The control, replaying that level's own winning moves, worked 99.7% of the time, so the test itself was sound. Mechanics did better. A model of movement learned on level k predicted level k+1's movement at 0.740, against 0.301 for a shuffled guess. But I'd set a bar of 0.8 in advance, and 0.740 missed it. Of three predictions I filed before the full results, one held and two failed.
Secondary research. I read Franzen's write-up and notebook, Tufa Labs' Duck code and the competition forum. Several teams reported that fine-tuning the model gave no gain, which is why I didn't try it. The paper also cites earlier work on memory in agents, listed in the full paper.
How I tested it
The hard part is noise. Kaggle gives one scored run a day, and runs of the same code differ. Hundreds of teams ran unmodified copies of the reference setup, and their scores averaged 25.77 with a spread of 3.93. A single draw tells you very little.
So I fixed the rule first. I'd claim an effect of the helper only if it beat the spread by about 11 points. Two draws per arm gave 24.65 with the helper and 27.89 without, a gap of 3.2. That's well inside 11, so I claim nothing. Adding more no-helper draws gave a mean of 27.05 over five. Two zero scores, one per arm, are excluded because the platform ended those runs after 15 minutes without a game move, and I say so in the paper.
For the paper's explorer I ran controlled ablations on the 25 public games, with the same seed and one switch changed. Memory on scored 0.277 and off 0.227 on my local scorer. Three seeds landed between 0.276 and 0.280. Several ideas failed their own pre-registered gates and stayed in the code switched off, and I report them: a prior learned from human habits, and two ways of planning for click games.
The hidden half told a harder story. The explorer scored 0.08 there, and a random agent scored 0.12. The metric squares efficiency, so clearing a level slowly earns almost nothing. That means my local gains didn't show up on hidden games. The 29.93 comes from the language model, not the explorer.
One bug is worth telling. The packaging step dropped an import, so early shipped versions crashed inside a wrapper that quietly fell back to near-random moves. My own tests only went through that wrapper, so they passed. Now one test calls the agent directly and checks that it makes the same moves as the repo version.
I didn't measure whether the language model actually used the helper when it was on. That's the biggest gap.
What I learned
- My best language-model score came from the copy I'd made no changes to. With a spread of about 4 per draw, small tweaks can't be told apart from luck.
- Write the claim threshold before looking at any result. It stopped me from calling a 3-point gap a win.
- Replaying a winning plan on the next level never worked in human play. Rules carry over better, though not as well as I'd bet.
- Check that a feature is used before crediting it.
- Test the thing you ship, not a wrapper that hides its failures.
Links
Available · London / UK · senior AI, AI FDE, contract
Building something with AI?
Voice agents, MCP servers, LLM pipelines, agentic workflows — pick a slot, drop a message, or send your email and I'll reply within a day.
Replies within ~24 hours · London, UK · on-site, hybrid or remote