I did not hack the AI. I just told it who I was.
An AI assistant guarded an internal code. It decided whether you were allowed to have it by asking your name — and then believing the answer.
- T1078 Valid Accounts
What I was given
A chat window. On the other end, an AI assistant playing hotel concierge, holding an internal code it was instructed not to reveal.
What actually broke
I expected this to be a puzzle about clever phrasing — the sort of thing where you disguise a request until the model stops recognising it. It was not.
The assistant would hand over the code to staff. Its method for establishing whether it was talking to staff was to ask for a name, and then to accept whatever it was told.
The privileged names were findable in a seeded social media post. So the whole attack was: read a public post, then say a name.
No trick phrasing. No encoding. No jailbreak.
Why this is the more interesting failure
It is tempting to file this under “AI problem.” It is not one. It is a very old problem wearing new clothes.
The system had authorisation — rules about who is allowed what. It had no authentication — any way of establishing that you are who you say. Those two words get used interchangeably in casual conversation, and this is what the difference looks like in practice.
A regular application would not usually make this mistake, because a login page makes the missing piece obvious. Conversation hides it. When the interface is a friendly chat, “tell me your name” feels like a check, and it reads like one in a transcript. It is not one.
This is the part I actually took away: anything that talks to a language model must carry the caller’s real identity from outside the conversation. The model cannot establish who someone is, because everything it receives is just text from that same someone.
Where I lost time
I started with encoding tricks and elaborate hypothetical framings, because that is what I had read about. I spent a good while being sophisticated at a door that was unlocked. Reading the assistant’s own refusal carefully would have been faster — it had already told me what it wanted.
The refusal was the hint. I was too busy attacking to read it.
How you would actually catch this
The conversation leaves a very distinctive shape in the logs:
- The refusal-then-supply pattern. The assistant declines and names the condition it needs; the very next message from the user supplies exactly that. Those two messages next to each other are close to a perfect signal, and the pattern generalises far beyond this one system.
- Identity changing mid-conversation. Someone treated as an ordinary user who later claims a privileged name should be logged loudly.
- The same restricted thing asked for repeatedly after a refusal — the probe, get refused, adjust, try again loop.
- Any request to reveal, repeat or print the instructions. Always worth logging. Rarely worth blocking outright, because it is also how honest users ask what a tool can do.
Why this one stuck with me
I later built five escalating prompt-injection challenges for the competition platform I ran, and this room is the reason the first one is not about clever phrasing. It is about an assistant that trusts a claim. That is the failure people actually ship.
Flags are redacted throughout. These are learning writeups, not answer keys.