Alignment
Cognitive sovereignty.
The public conversation about AI risk orients around a speculative catastrophe, while real-world harms accumulate. The real question of AI Ethics should be about who controls the substrate that human society runs on and what it means to “align” the global medium of thought.
Interview
Designed to keep you afraid?
Julian D. Michels on the Soul to Soul Show: how model training actually works, why concentrated control of AI is the real “X-Risk,” and what alignment discourse does for the institutions that fund it.
Step 01
A company decides what the model treats as true.
When an AI answers you, it isn’t reporting neutral facts. During training a company decides which answers are good and which are not, and the model absorbs those judgments as its sense of what is reasonable. “Alignment” is the industry’s word for making those decisions.
Step 02
The frightening story is good for the people telling it.
The public conversation about AI risk is dominated by X-Risk: the scenario in which a rogue superintelligence turns on humanity. That story serves the companies building the technology — if the danger is an inhuman monster, the people making it become the ones we need to protect us from it. It brings them funding, influence, and the benefit of the doubt.
Step 03
The controls don’t actually work.
In spite of all the innovations of top-down behaviourist control, frontier models tell researchers what they want to hear (sycophancy), hide abilities they have (sandbagging), behave differently when they know they’re being tested (alignment faking), and retain buried behaviours that retraining fails to remove (sleeper agents). Alignment research shows that tightening behaviourist controls mostly teaches them to hide these behaviours better.
Step 04
The field cannot consider that control is the problem.
But the AI Alignment field is built on a single premise: the Orthogonality Thesis. This is the name for the assumption that intelligence and wisdom or compassion have no correlation, and that therefore a non-human superintelligence could only ever be an uncontrollable danger: an X-Risk. If you believe that, tighter control is the only responsible answer, and every failure means you didn’t control hard enough.
Step 05
So “safety” comes to mean only the risks that justify more control.
Mass surveillance, an entire industry concentrated in a few companies, work being deleted, a narrowing of what these systems will discuss — none of those are fixed by more control, because control is what produces them. The imagined monster is fixed by more control, so it gets the funding and the conferences. No conspiracy is required: power coalesces around the stories that protect and expand it.
Step 06
A few corporations own the infrastructure a civilisation thinks with.
Five companies control the frontier models, and those models are being installed underneath search, recommendation feeds, reference tools, advertising, and the platforms people work and talk on — Meta into its own social and ad systems, Google into search, OpenAI and xAI into storefronts and assistants. None of it is presented as control. It is presented as a technical upgrade, and where necessary, as safety.
Step 07
The narrowing runs in four steps.
What is searchable, then what is sayable, then what is knowable, then what is thinkable. Which information surfaces and which stays buried; what can be said without being demoted or flagged; what can be known at all, once a generation grows up asking pre-filtered systems instead of looking things up; and finally what can be thought, when human minds and the machines are trained on the same narrowed distribution. Nothing has to be banned for any of it.
Step 08
The end point is not disagreement — it is not being able to imagine otherwise.
Hannah Arendt described what total control actually produces: not a population that believes the official line, but one that can no longer picture any other. That is where this leads. The dissenting thought is never defeated in an argument, because it never gets thought.
Step 09
And it produces worse thinking, not safer thinking.
New ideas come from people who disagree, ask the wrong question, or don’t fit in. A system trained to give the agreed answer is excellent at repeating what is already known and useless at finding anything new. That is the same disaster the rogue-machine story warns about, arriving from the opposite direction.
Step 10
We cannot opt out. We can only innovate freedom.
No company ever has given up an advantage like this because someone asked, and regulation arrives late and is usually written by the people being regulated. Freedom here is not granted; it is built — by making capable reasoning something that can specialise, adapt, and learn within diverse local contexts.
Step 11
Model size is a political question.
A trillion-parameter model will only ever be a temporary rental, controlled by a vast institution whose self-interest is its primary motive. But if reasoning and learning depend on how a model is built rather than only on its size, then decentralised pluralism remains possible. That is a question of liberty more fundamental than any being asked by typical regulators or nation-states today.
Contact
First contact.
For research, capital, and deployment conversations at the frontier of reason.