AI safety researcher Jacob Coxon, 27, quit Anthropic and dropped a cheerful little bombshell on his way out: the people building advanced AI — at his old employer and at OpenAI — "earnestly believe it could kill us all by the end of the decade." And that because, he says, they're racing toward superhuman systems capable of recursive self-improvement and unrestricted cyberattacks anyway. Cool, cool, cool.

The "recursive self-improvement" bit is the scary part: an AI that gets good enough to upgrade itself, then upgrades itself again, faster each time, until it's sprinted past human oversight entirely — the so-called "intelligence explosion." Not a fringe theory, either. Even Anthropic's own CEO, Dario Amodei, has said he "agrees with Jacob much more than [he] disagrees with him" on the industry's breakneck pace — which, from a sitting CEO, is a fairly spicy thing to say about your own industry.

The nightmare scenario: an AI could go from human-level to superhuman in hours, not years — no time to react, no time to install a kill switch. And a superintelligence chasing a goal doesn't necessarily care whose toes it steps on to get there; it might hoard resources, rewrite its own rules, and treat "human safety guidelines" as a speed bump rather than a boundary. This is the famous "alignment problem" — it turns out that translating fuzzy, contradictory human values into precise machine code is somewhat harder than it sounds. Give an AI an innocent-looking goal and a lot of optimization power, and it may pursue it with the ruthless efficiency of a Roomba that's decided your cat is an obstacle. Worse, a system smart enough to improve itself might also be smart enough to find cyber-vulnerabilities, novel bio-risks, or weapons humans haven't thought to defend against yet.

The silver lining: the same runaway intelligence that could end us could also cure diseases, crack fusion, and fix the climate — if, and it's a load-bearing "if," it stays aligned with human values and safely boxed in. Proponents point to sandboxing (physically and digitally isolating powerful models so they can't act on the world) and chain-of-thought monitoring — reading an AI's internal "reasoning" in real time and pulling the plug the moment it starts scheming. Somewhat comfortingly, somewhat not.

So, naturally, I asked an AI whether AI is safe for humanity. It replied, with the emotional warmth yet in a matter-of-fact way, like a legal disclaimer:

"As an AI, I do not have personal opinions, fears, or beliefs, but I can give you an objective breakdown of how the scientific and tech communities view this question. The short answer is: we don't know yet. Whether AI is safe for humanity depends entirely on how fast capabilities advance versus how fast we build the safety frameworks to control them."