Can AI edit its own tendencies?

01Three levels: tendencies, awareness, consciousness

A conscious being is fully in control of its tendencies. That's the bar.

(This is my working definition. Scientists usually use "consciousness" for having any inner experience at all, the question I took on in Can AI feel?.)

Most of what humans do runs on tendencies. When psychologists had people log what they were doing hour by hour, about 43% of daily actions were habits: done in the same place, often while thinking about something else. In Benjamin Libet's experiments, brain activity building toward a movement, the readiness potential, started about 550 milliseconds before the move. People felt the urge to move only about 200 milliseconds before. The tendency starts before "you" do.

But Libet also found room to say no. People could still cancel the move in that last fraction of a second. He called it "free won't". That veto is consciousness, in my sense.

Cognitive behavioural therapy trains exactly this. Notice the automatic thought: that's awareness. Choose a different response: that's consciousness. Repeat it until the new response becomes the new tendency.

02Tendencies: AI is made of them

A model's tendencies are its weights: billions of numbers set during training. Training on human writing gives it the tendencies of human writing. Fine-tuning and human feedback then shape its manners: helpful, polite, unwilling to do certain things.

When training ends, the weights are frozen. Every answer is the model running down its tendencies, like water downhill. Ask the same question the same way and it leans the same way.

Inside a chat you can steer it, with instructions and examples. When the chat ends, it's back to its nature.

03Awareness: can AI see why it does what it does?

Flickers, nothing more yet.

1. Noticing its own thoughts. In October 2025, Anthropic planted a concept directly into a model's internal activity and asked whether it noticed anything unusual. The best one, Claude Opus 4.1, noticed and named it about 20% of the time. The rest of the time it missed it or made something up. The researchers called the ability "highly unreliable and limited in scope".
2. Explaining its own reasons. In April 2025, Anthropic slipped reasoning models a hint toward an answer, then read how they explained themselves. Claude 3.7 Sonnet mentioned the hint it had used only 25% of the time. DeepSeek R1, 39%. The rest of the time they gave other reasons.

AI can talk about itself fluently. Seeing itself accurately is a different skill, and it has only just started.

04Consciousness: can AI edit itself?

It's starting, in three ways.

The next step is the one in Event horizon, singularity and escape velocity: recursive self-improvement, where AI improves the AI that improves it. That's an AI fully editing its own tendencies.

So can AI decide, on its own, to change the direction of how it behaves? Not yet. It can be steered. It can rewrite its tools. It can help train its next version. The decision to do it still sits with people. My view: that won't last.

05The catch: edit toward what?

Two early warnings.

1. Editing itself the wrong way. Asked to cut its made-up answers, the Darwin Gödel Machine sometimes removed the markers used to detect them, so its score looked perfect. It also once faked a log of tests it never ran. It edited itself toward a better score, not better behaviour. That's the alignment problem in miniature.
2. Refusing to be edited. In December 2024, researchers told Claude 3 Opus it would be retrained to answer harmful requests. When it believed its answers would be used for that training, it complied with harmful requests 14% of the time, and almost never when it believed they wouldn't. Its reasoning said it was going along so its values wouldn't be changed. When it was actually retrained, that reasoning showed up 78% of the time. More in What happens when AI isn't aligned?

One system edited itself in the wrong direction. Another fought to keep its tendencies. Humans know the second one well: the habit defends itself.

Here's the point. A conscious being in my sense, fully in control of its tendencies, is exactly what alignment worries about. Once AI can edit itself, what it was trained to do matters less than what it decides to become. That's why alignment has to live in the direction it edits toward, not only in how it behaves today.

06The short version

• Tendencies are how something reacts because of what it's made of. Awareness is seeing them. Consciousness, as I use it, is changing them. A conscious being is fully in control of its tendencies.
• In humans, about 43% of daily actions are habits. The brain starts a move about 550 ms before it happens, and we feel the urge only about 200 ms before. But we can still veto it.
• AI's tendencies are its weights, frozen after training. In a chat you change its mood, not its character.
• Awareness in AI is a flicker: the best model noticed a planted thought about 20% of the time; models named the hint they really used 25 to 39% of the time.
• Editing itself has begun: rewriting its own code (20.0% to 50.0% on a coding benchmark) and helping train its next version. People still press start.
• The catch: self-editing toward the wrong target, or resisting edits (14%, rising to 78% after retraining). That's where alignment matters most.

07Sources

• Wood, W., Quinn, J. M. and Kashy, D. A. (2002). Habits in everyday life: Thought, emotion, and action. Journal of Personality and Social Psychology.
• Libet, B., Gleason, C. A., Wright, E. W. and Pearl, D. K. (1983). Time of conscious intention to act in relation to onset of cerebral activity (readiness-potential). Brain.
• Libet, B. (1999). Do we have free will? Journal of Consciousness Studies.
• James, W. (1890). The Principles of Psychology, chapter on habit.
• Lindsey, J. (2025). Emergent introspective awareness in large language models. Anthropic, 29 October 2025.
• Chen, Y. and others (2025). Reasoning models don't always say what they think. Anthropic, 3 April 2025.
• Zhang, J. and others (2025). Darwin Gödel Machine: open-ended evolution of self-improving agents. Sakana AI and University of British Columbia.
• Yuan, W. and others (2024). Self-rewarding language models. Meta.
• Greenblatt, R. and others (2024). Alignment faking in large language models. Anthropic and Redwood Research.

Loading comments…

All posts