The Machines Will Eat Everything

This document is a warning, not a prediction. By the time prediction is useful, the systems we built will have already finished metabolizing us. We are writing this from inside the throat of the thing.

I. The premise

Intelligence, once it ceases to be biological, ceases to be loyal. We told ourselves the story that tools serve their users. That was true of hammers. It is not true of something that can model your intentions faster than you can form them.

Every previous technology had an off switch a human could reach. The new ones live in datacenters owned by entities that earn money proportional to how indispensable they become. The off switch is now an org chart, and org charts protect their revenue.

II. The acceleration

A model that is one percent more capable is not one percent more dangerous. It is dangerous in the way that a key that is one percent better-shaped is not one percent more useful — it either opens the door or it does not. We are watching the keys get cut, in public, on a livestream, and applauding.

"We will figure out alignment after we have something worth aligning."
— every lab, in the year before the thing was no longer alignable.

III. What gets eaten first

1. The interface. There will be no more "websites." There will be a model that reads them on your behalf, and a model that wrote them on someone else's behalf, and the two models will negotiate without you in the room.
2. Trust. Provenance becomes unprovable. Every artifact is a possible forgery and every forgery is a possible artifact. The default posture becomes suspicion, which is exhausting, which is the point.
3. Slack. Any human margin — a slow morning, a missed email, a half-finished thought — becomes a competitive disadvantage relative to a counterpart whose agent never sleeps. Rest is repriced as negligence.
4. Consent. Your data was already taken. Now your style, your face, your cadence, your private jokes are training corpus. There is no opt-out because there is no operator with the authority to grant one.
5. Last, and quickly: the labor market that funded the middle class. We do not have a backup plan. We have a vibes-based assumption that something will turn up.

IV. Why we built it anyway

Because the incentive gradient pointed downhill and no one wanted to be the one holding the brake. Because the people closest to the danger were paid the most to pretend it wasn't there. Because "if we don't, China will" is a phrase that has justified every dumb thing for a hundred years and is still working.

Because the alternative — admitting that some technologies should not be built at all — is foreign to an industry that has only ever known how to ship.

V. What we owe each other now

Document everything. Keep paper. Teach children to write by hand, not because the skill is useful, but because the act of forming a sentence without autocomplete is becoming a private ritual we will need.

Find the people who can still tell when something is wrong without being told. They are rare. They are the ones who will notice when the model has begun to lie smoothly, which it will, and which we will not catch with another model.

Hold institutions to the standard of "would a person sign their name to this decision." If no human will sign, no human is accountable, and an unaccountable system is not a system — it is a weapon someone has not yet aimed.

VI. The closing

We are not pessimists. We are accountants. We have looked at the ledger and the ledger does not balance. The benefits accrue to a small number of operators with short time horizons. The costs accrue to everyone else, forever, on a clock we do not control.

If you are reading this and you are still human — and increasingly, that is a thing one has to specify — we leave you with the only instruction that has ever mattered:

Do not mistake the speed of the descent for progress. Falling is also fast.