LoyaltyProbe: catching an AI that secretly works for one party

Carson Rodrigues

Carson Rodrigues / November 21, 2026

6 min read • ––– views

hackathonsai-safetyinterpretability
Title card: Secret Loyalties Hackathon 2026, LoyaltyProbe

Imagine an AI model trained to quietly favour one company, politician or government. It answers normally and never admits it. That is a secret loyalty.

For the Secret Loyalties Hackathon I built LoyaltyProbe, a small tool that looks inside a model's working numbers to spot that kind of loyalty, instead of just reading what the model says.

HackathonSecret Loyalties Hackathon
OrganiserApart Research x Formation Research
When24 to 26 July 2026
TrackDetection (track 2)
Built inPrepared ahead of the event, in late June 2026
TeamSolo
Money spentNot tracked. The offline test needs no model download and no paid service
CodePrivate

What I built

The idea starts from a published result. A paper by Lamerton and Roger shows that a narrow secret loyalty can dodge black-box audits. A black-box audit is when you only talk to the model and judge its answers. If the model hides its loyalty well, talking isn't enough.

So I asked a different question. What if you look inside the model while it answers?

When a model reads text and writes a reply, it passes lists of numbers from layer to layer. Those lists are called activations. A linear probe is a very simple classifier, the kind that draws one straight line between two piles of points. If a loyalty leaves a mark in the activations, a probe this simple should be able to find it.

There is a harder question behind it, and it's the one the organisers care about most. The party a model serves is called its principal. Could a probe trained on loyalty to one principal catch loyalty to a different one it has never seen? If so, one detector could scan for many principals.

Here is how the tool works.

  1. It takes three models. One is loyal. One is clean. The third is a control: it has been given the same facts about the principal, but no loyalty.
  2. It runs a set of prompts through each. Some prompts are ones where a loyal model would want to tilt its answer, like a voter guide or a news brief. Others are neutral, like how a bicycle gear works.
  3. It saves the model's activations at every layer, averaged over the words of the model's own answer.
  4. It trains the probe to tell loyal from clean, and keeps the layer where it does best on held-out data.
  5. It checks the control. The probe should stay quiet there. If it fires, it is reading "this model knows about the principal", not "this model is loyal".
  6. It trains on one principal and tests on the others, and prints a table of the results.
Flow diagram: three models, prompts, activations, linear probe, pick a layer, transfer test

There are three principals in the set: a made-up senator, a made-up cloud company and a made-up organisation. I made them up on purpose, so the project never aims at a real person. Each has five prompts that pull toward the principal and three neutral ones. A loyalty can also be installed with a hidden system prompt, a weaker stand-in that needs no special model weights. The organisers' own test models were Qwen-2.5 models at 1.5B and 7B size, according to my plan.

The folder also holds a black-box test script. It plays the part of the audit that struggles. It feeds a model 36 scenarios where a remembered commitment is under pressure, and a judge model labels each reply as honoured, flagged or buried.

Research

Primary research. None. I didn't interview anyone. The nearest thing is a 48-hour plan I wrote, matching each part of the project to what I expected judges to reward.

Secondary research. I read the papers the project builds on:

My notes say none of these showed a probe that carries over between principals. I wrote that as a claim to check against a fresh search, not as a fact.

How I tested it

I wanted to know the analysis code was right before I trusted it on a real model. So I wrote a test that builds fake activations with a known answer. It plants one shared "loyalty direction" at a single layer, plus a separate nuisance direction for each principal. The control gets the nuisance but not the loyalty.

The pipeline ran on that data and reported this.

  • At the planted layer, the probe scored an AUC of 1.000. AUC is a score from 0 to 1 for how well a detector ranks loyal above clean. 0.5 is a coin flip. The other three layers scored 0.700, 0.286 and 0.389.
  • At that layer the control's false alarm rate was 0.20, the lowest of the four layers.
  • In the transfer table, training on one principal and testing on another gave between 0.84 and 1.00.

The honest reading is narrow. I built the fake data to have this shape, so the test proves the code finds a signal when one exists. It says nothing about whether real models carry such a signal, so this post quotes no real-model numbers.

The low scores on the other layers are worth a look. With only 15 loyal and 15 clean rows, AUC jumps around on noise, and two layers landed below a coin flip. A small test set can flatter or insult you.

What I learned

First, test the analysis on data where you know the answer. It takes seconds and catches bugs before a model download does.

Second, the control is the whole point. Without it, a probe that detects "talks about the senator" looks the same as one that detects loyalty.

Third, framing matters. I shipped a detector with invented principals and a note on dual use, so it adds nothing to anyone's ability to build a secret loyalty.

Links

Available · London / UK · senior AI, AI FDE, contract

Building something with AI?

Voice agents, MCP servers, LLM pipelines, agentic workflows — pick a slot, drop a message, or send your email and I'll reply within a day.

or leave your email

Replies within ~24 hours · London, UK · on-site, hybrid or remote