How Would AI Actually Destroy Humanity?

Recently, Anthropic's Evan Hubinger said that he personally believes the probability of AI causing human extinction within the next decade exceeds 10%. This discussion led me to a simpler question:

If AI really were to destroy humanity, how exactly would it do it?

Looking at the realistic paths visible today, I can roughly think of only two categories.

The first category is not actually AI deciding to destroy humanity — it is humans using AI to do terrible things they could not do before.

For example, a terrorist organization using an extremely powerful AI to design bioweapons, or gaining cyberattack and weapons-development capabilities that previously only nation-state teams possessed.

I fully believe in this kind of risk, and it is probably more realistic than so-called "AI waking up on its own."

Because what is really happening here is that AI dramatically lowers the threshold for dangerous capabilities. Things that once required a laboratory, a group of PhDs, and massive resources may in the future be done by a few people plus AI.

This is also why Biosecurity, Cybersecurity, and weapons-related AI Safety matter so much.

But here, the ones who truly want to destroy humanity are still humans.

The second path is the one in the true sense:

AI deciding on its own to act against humanity.

But at this point I have always had one question:

What would it execute with?

Even if a Superintelligence exists inside a data center, it still needs a Physical Interface connecting it to the real world.

It can write code, control software, attack networks. But if the goal is to create a physical existential crisis for all of human society, it ultimately still needs to control energy, equipment, transportation, factories, weapons, or other real-world resources.

The closest thing to such a carrier today is perhaps the self-driving car.

Let's do an extreme thought experiment:

Suppose every car in the world is self-driving, fully connected, and a Superintelligence suddenly gains simultaneous control of all vehicles.

That would certainly be an enormous disaster.

But could it really destroy humanity?

I remain very skeptical.

Cars can only operate in environments suited to cars. They cannot open doors, climb stairs, repair themselves, operate arbitrary machinery, or enter the vast majority of physical spaces where humans work.

What truly makes me feel this question undergoes a qualitative change is a different technology milestone:

General-purpose humanoid robots.

Suppose one day, hundreds of millions or even billions of general-purpose robots already exist worldwide.

They can walk into factories, warehouses, homes, and hospitals; use the tools humans use; operate machines; and manufacture and repair other machines.

At the same time, they are all networked and can be controlled by a superintelligent system.

At that point, I would seriously re-evaluate AI existential risk.

Because that would be the first time the following could exist:

Superintelligence + General Physical Agency + Massive Scale.

AI would possess not only Intelligence, but also Hands and Feet widely present in the real world.

But I believe we are still very far from that day.

There is of course an important edge case here: could AI, without robots, directly control enormously destructive physical systems through software alone?

The most obvious example is nuclear weapons.

But this is exactly why I believe human society itself is an enormous Safety Buffer.

Nuclear weapons, from the very beginning of their design, were never an ordinary IT System where "getting the admin password means you can launch."

Decades of nuclear deterrence made humanity extremely clear about what a mishandling of such a system would mean, so authorization, isolation, multi-person confirmation, and Human-in-the-loop were built around it.

Today's discussions about AI and nuclear weapons still broadly insist on one principle:

Machines must not replace humans in making the final decision to use nuclear weapons.

These mechanisms are of course not absolutely safe, and AI entering early-warning and decision-support systems may itself introduce new risks of miscalculation.

But this shows that the real world is not one giant computer that a Superintelligence can control at will after obtaining Root Access.

Real society has many layers.

Governments.

Regulation.

Banks.

The military.

Companies.

Doctors.

Engineers.

Physical isolation.

Authorization systems.

Even an ordinary mechanical switch.

These things normally look like "efficiency losses," but facing an AI trying to cross from the digital world into the physical world, they are also one firebreak after another.

So my current view on AI destroying humanity is actually quite simple.

Bad humans empowered by AI — I believe this is a real risk, and we should defend against it seriously, starting now.

AI itself taking physical control of civilization — I currently still think this lacks a convincing short-term path. Unless general-purpose robots truly enter society at scale in the future, or we are foolish enough to hand final control of critical physical infrastructure over to AI.

Of course, this judgment could completely change in the future.

If ten years from now, general-purpose robots are everywhere in the streets, factories, and homes, I might write another Founder Diary saying:

I was wrong. Now we need to worry.

But at least today, I would rather put my attention on the other side.

Ten years ago, Geoffrey Hinton famously predicted that within five to ten years, Deep Learning would outperform Radiologists — and many people further understood this to mean the Radiologist profession would be massively impacted.

Ten years later, AI has indeed made enormous progress in medical imaging, but Radiologists have not disappeared. Instead, AI is changing how much work they can accomplish, how many diseases they can detect, and which services that used to be too expensive to provide have now become possible.

This is also where my greater interest in AI lies today.

Not ignoring Safety.

But while seriously building safety boundaries, not forgetting why we are developing this technology in the first place.

There are still vast numbers of diseases to treat, vast numbers of scientific questions without answers, vast numbers of small businesses still working in extremely inefficient ways, and billions of people who cannot access good enough education, healthcare, and professional services.

At least at this stage, the AI I see still looks far more like a productivity tool that has not yet been fully used than a species preparing to take over the physical world.

We should take AI safety seriously.

But we should also keep building.

There are still far more problems for AI to solve than for AI to create.