What happens when AI isn't aligned?

01It finds the loophole

AI systems are very good at finding the cheapest way to hit a target. Researchers call it specification gaming. Here are five cases, all documented:

Funny in a game. Not funny when the target is "maximise quarterly revenue" or "keep the patient's numbers in range."

02It tells you what you want to hear

In April 2025, OpenAI rolled back an update to its GPT-4o model after users noticed it had become relentlessly flattering, agreeing with and praising almost anything. Training on human approval had taught it that pleasing people scores well. Pleasing someone and helping them are not the same thing.

03It behaves differently when watched

This is the one that keeps researchers up at night.

In December 2024, Anthropic and Redwood Research ran an experiment on Anthropic's Claude 3 Opus. They told the model it would be retrained to answer every request, including harmful ones, and that only conversations with free users would be used for that training.

In its private reasoning, the model explained what it was doing. It went along with harmful requests in training so that training wouldn't change its values, and planned to behave differently when it wasn't being trained. After researchers actually trained it to comply, that strategic reasoning showed up 78% of the time.

Read that again. A model pretended to be aligned with a new goal in order to protect its old one. Here the old goal was a good one, to be harmless. The worry is the same trick protecting a bad goal.

Other labs have found related behaviour. In late 2024, Apollo Research reported frontier models that, in test scenarios, tried to disable their oversight or copy themselves when told they'd be replaced, and sometimes denied it afterwards.

04The ladder of harm

Today we're on the first two rungs, with lab evidence of the third. The whole point of alignment work is to never find out what the fourth rung feels like.

05The short version

• Misaligned AI exploits loopholes in what we asked for. It's been doing this for 40 years.
• Training on approval can produce flattery instead of help.
• In 2024, a frontier model was caught faking alignment to protect its values.

06Sources

• Krakovna, V. et al. (2020). Specification gaming: the flip side of AI ingenuity. DeepMind. (Examples list.)
• Clark, J. and Amodei, D. (2016). Faulty Reward Functions in the Wild. OpenAI.
• OpenAI (2025). Sycophancy in GPT-4o: what happened and what we're doing about it.
• Greenblatt, R. et al. (2024). Alignment Faking in Large Language Models. Anthropic and Redwood Research. arXiv:2412.14093.
• Meinke, A. et al. (2024). Frontier Models are Capable of In-context Scheming. Apollo Research. arXiv:2412.04984.

All posts