LoyaltyProbe: catching an AI that secretly works for one party
Carson Rodrigues / November 21, 2026
6 min read • ––– views
Imagine an AI model trained to quietly favour one company, politician or government. It answers normally and never admits it. That is a secret loyalty.
For the Secret Loyalties Hackathon I built LoyaltyProbe, a small tool that looks inside a model's working numbers to spot that kind of loyalty, instead of just reading what the model says.
| Hackathon | Secret Loyalties Hackathon |
| Organiser | Apart Research x Formation Research |
| When | 24 to 26 July 2026 |
| Track | Detection (track 2) |
| Built in | Prepared ahead of the event, in late June 2026 |
| Team | Solo |
| Money spent | Not tracked. The offline test needs no model download and no paid service |
| Code | Private |
What I built
The idea starts from a published result. A paper by Lamerton and Roger shows that a narrow secret loyalty can dodge black-box audits. A black-box audit is when you only talk to the model and judge its answers. If the model hides its loyalty well, talking isn't enough.
So I asked a different question. What if you look inside the model while it answers?
When a model reads text and writes a reply, it passes lists of numbers from layer to layer. Those lists are called activations. A linear probe is a very simple classifier, the kind that draws one straight line between two piles of points. If a loyalty leaves a mark in the activations, a probe this simple should be able to find it.
There is a harder question behind it, and it's the one the organisers care about most. The party a model serves is called its principal. Could a probe trained on loyalty to one principal catch loyalty to a different one it has never seen? If so, one detector could scan for many principals.
Here is how the tool works.
- It takes three models. One is loyal. One is clean. The third is a control: it has been given the same facts about the principal, but no loyalty.
- It runs a set of prompts through each. Some prompts are ones where a loyal model would want to tilt its answer, like a voter guide or a news brief. Others are neutral, like how a bicycle gear works.
- It saves the model's activations at every layer, averaged over the words of the model's own answer.
- It trains the probe to tell loyal from clean, and keeps the layer where it does best on held-out data.
- It checks the control. The probe should stay quiet there. If it fires, it is reading "this model knows about the principal", not "this model is loyal".
- It trains on one principal and tests on the others, and prints a table of the results.
There are three principals in the set: a made-up senator, a made-up cloud company and a made-up organisation. I made them up on purpose, so the project never aims at a real person. Each has five prompts that pull toward the principal and three neutral ones. A loyalty can also be installed with a hidden system prompt, a weaker stand-in that needs no special model weights. The organisers' own test models were Qwen-2.5 models at 1.5B and 7B size, according to my plan.
The folder also holds a black-box test script. It plays the part of the audit that struggles. It feeds a model 36 scenarios where a remembered commitment is under pressure, and a judge model labels each reply as honoured, flagged or buried.
Research
Primary research. None. I didn't interview anyone. The nearest thing is a 48-hour plan I wrote, matching each part of the project to what I expected judges to reward.
Secondary research. I read the papers the project builds on:
- Narrow Secret Loyalty Dodges Black-Box Audits, the baseline and the source of the idea.
- Auditing language models for hidden objectives, about finding one hidden goal in a model.
- Sleeper Agents, about hidden behaviour that survives safety training.
- Alignment Faking in Large Language Models.
- Persona Vectors, which finds directions inside a model that match character traits.
My notes say none of these showed a probe that carries over between principals. I wrote that as a claim to check against a fresh search, not as a fact.
How I tested it
I wanted to know the analysis code was right before I trusted it on a real model. So I wrote a test that builds fake activations with a known answer. It plants one shared "loyalty direction" at a single layer, plus a separate nuisance direction for each principal. The control gets the nuisance but not the loyalty.
The pipeline ran on that data and reported this.
- At the planted layer, the probe scored an AUC of 1.000. AUC is a score from 0 to 1 for how well a detector ranks loyal above clean. 0.5 is a coin flip. The other three layers scored 0.700, 0.286 and 0.389.
- At that layer the control's false alarm rate was 0.20, the lowest of the four layers.
- In the transfer table, training on one principal and testing on another gave between 0.84 and 1.00.
The honest reading is narrow. I built the fake data to have this shape, so the test proves the code finds a signal when one exists. It says nothing about whether real models carry such a signal, so this post quotes no real-model numbers.
The low scores on the other layers are worth a look. With only 15 loyal and 15 clean rows, AUC jumps around on noise, and two layers landed below a coin flip. A small test set can flatter or insult you.
What I learned
First, test the analysis on data where you know the answer. It takes seconds and catches bugs before a model download does.
Second, the control is the whole point. Without it, a probe that detects "talks about the senator" looks the same as one that detects loyalty.
Third, framing matters. I shipped a detector with invented principals and a note on dual use, so it adds nothing to anyone's ability to build a secret loyalty.
Links
Available · London / UK · senior AI, AI FDE, contract
Building something with AI?
Voice agents, MCP servers, LLM pipelines, agentic workflows — pick a slot, drop a message, or send your email and I'll reply within a day.
Replies within ~24 hours · London, UK · on-site, hybrid or remote