I built a continuity checker for film sets. It missed three in four of the errors I planted.

Carson Rodrigues

Carson Rodrigues / November 21, 2026

6 min read • ––– views

hackathonsai-agentsfilm
Scripty's evaluation page: 6 scenes, 45 planted errors, 27% recall, 100% precision, and a table of results by kind of error

On a film set, one person has to notice that the wine glass was half full in one take and empty in the next, or that the tie was knotted in the wide shot and loose in the close-up.

That person is the script supervisor, and everyone calls them "scripty". They do it with photos, a marked-up script and years of practice.

HackathonAgentic Cinema, ClickHouse track
OrganiserDevpost hackathon, with Google Cloud and ClickHouse as partners
WhenSubmitted 9 September 2026
Result and rankDid not win (confirmed 17 October 2026). My notes record no placing or field size
Built inAbout four days, from 03:40 IST on 5 September
TeamSolo
Money spentNot tracked. My notes put the always-on database service at about $39 a month at list price; the AI model and other cloud costs weren't recorded
Codegithub.com/rodriguescarson/scripty (Apache-2.0)

The hackathon

Agentic Cinema had a $75k prize pool, split across five tracks. I entered the ClickHouse track, which is run with Google Cloud. ClickHouse is a database built to answer questions over huge piles of rows fast.

The rules were strict. The project had to be new, built inside the window, use Google's AI at runtime, and also use the partner's tool at runtime. It needed a public repo and a video of three minutes or less.

What Scripty does

You give it several takes of one scene. It samples frames from each one and asks an AI model to write down everything it can see, as plain rows: the object, what's true about it, and its value. "Man's tie, colour, orange."

Then the database does the part it's good at. It lines up every row and finds the things that changed between takes or shots of the same scene.

Flow diagram: takes, sampled frames, AI reads each frame, database finds changes, a stronger AI rules on each change, shortlist with both frames

A second look from a stronger model decides each change: a real continuity error, an intentional change, or the same thing. A glass that's emptier because the actor drank from it isn't a mistake.

The supervisor gets a short list with both frames side by side. There's also a chat agent that answers questions like "what changed on the desk between take A and take C?", and it can only reach the data through ClickHouse's official connector.

Research

Primary research. I built my own test set from a public-domain film, Charade (1963), because no open set of continuity errors exists. I ran pilot scenes first, and they taught me things no reading would have: the model named the same object differently from one take to the next, and its confidence score told me nothing. I didn't interview any script supervisors. My notes record no user interviews.

Secondary research. Before building I looked at prior attempts: one academic paper from 2009 and three shelved patents from companies that hit the same wall. I found nothing open source. All of them stall on the same problem, telling an intentional change from a mistake. I recorded no study of past winners of this hackathon.

How I tested it

I wrote the pass mark down before I ran anything, at 03:55 IST on 5 September. I made two extra copies of each scene. A control copy had small harmless changes (re-framing, re-exposing) and nothing else. A planted copy had one error per shot, put in on purpose: a prop removed, moved or recoloured, or the frame mirrored. The planting log was the answer key.

Pass meant finding at least 60% of the planted errors, being right on at least half of what it flagged, and raising no more than one false alarm per scene on the control copy.

The plan was to score every scene that passed the filter, 31 of them, not a hand-picked few. I scored 6. The final run used a fixed rule to pick scenes, not my own choice after seeing results. I also checked every planted box by cropping it and asking a second model whether it showed the right object, and I blurred the station logo in the corner of every frame so a mirrored logo couldn't give the flip away.

There were also nine small automated tests on the plumbing: the query that finds changes, shot detection, the planting, and a budget check.

What I didn't have is held-out data. I fixed four problems after looking at one scene from the first run, then re-scored the same labels and frames. So the second-run number is a fair measure of the fixed pipeline, not a clean test of it. I labelled it as a second run everywhere.

What happened

It didn't win. I confirmed that on 17 October 2026, and my notes record no placing and no count of entries.

It also failed my own test. On 6 scenes and 45 planted errors, it found 27%. Everything it flagged was real (precision 100%), and the control copy raised about one false alarm every three scenes. So it was careful, but it missed three in four.

It was worst on colours: it caught 1 of 14 recoloured objects. Mirrored frames, which it missed completely on the first run (0 of 19), it caught 8 times out of 22.

Two more caveats. That 27% is from the second run. With a stricter way of counting a hit, it drops to about 16%. The first run, on 4 scenes, found 39%, and it raised nearly three false alarms per scene on the control.

What I learned

The test was worth more than the detector. Every early run looked good until a number said otherwise. The model's confidence score was 1.0 for almost everything, including wrong answers. A control copy where nothing had changed still produced false alarms. None of that shows up in a demo video.

Names drift. Asked twice about the same shot, the model gave the same name to only 35% of objects. The fix came from how a real script supervisor works: the first take becomes the reference sheet for every later one. False alarms on the control copy fell from 2.75 to 0.33 per scene.

And one scene is not evidence. A single-scene rerun gave me 100% recall, and I almost reported it. Across all six scenes it was 27%.

I also don't know why recolouring got worse between the two runs. The change that fixed mirrored frames seems to have broken it, and I haven't explained that yet.

Links

The hosted app is gone. I deleted the cloud services on 17 October so nothing keeps billing, and left a static snapshot with the scenes, findings and evaluation page.

Available · London / UK · senior AI, AI FDE, contract

Building something with AI?

Voice agents, MCP servers, LLM pipelines, agentic workflows — pick a slot, drop a message, or send your email and I'll reply within a day.

or leave your email

Replies within ~24 hours · London, UK · on-site, hybrid or remote