Sometime around late 2009, centrifuges at Iran’s Natanz uranium enrichment plant began malfunctioning. They would suddenly speed up or slow down, placing enormous stress on the delicate machinery. Centrifuges failed. The engineers trying to diagnose the problem had another problem: According to the computers monitoring the equipment, everything looked normal. The computers, in a way, were engaged in a cover-up.
A piece of malware later labeled Stuxnet had infiltrated the plant’s industrial control systems. It could recognize the particular equipment used at Natanz, alter the operation of the centrifuges, and, in one version of the attack, record normal readings and play them back to operators while the sabotage was underway. The engineers were looking at a recording of a healthy system while the real one was being damaged.
Stuxnet was unlike anything security researchers had seen before . It was enormous and extraordinarily sophisticated, exploiting multiple previously unknown Windows vulnerabilities and using stolen digital certificates to disguise itself as legitimate software. It could also spread through infected USB drives, allowing it to jump the “air gap” separating sensitive industrial computers from the Internet. Someone would plug a thumb drive into an infected computer, carry it into an isolated facility and plug it in there, infecting the network.
Stuxnet sounds almost alive when you describe it in action.
Once inside a new computer, however, it didn’t simply start breaking things. Stuxnet searched for Siemens industrial-control software, then checked the machinery attached to it for a very particular configuration of equipment. If the conditions weren’t right, it waited. Only when it found the system it had been built to recognize did the sabotage begin.
Stuxnet sounds almost alive when you describe it in action. It traveled extensively, looking for its target. It recognized it. It waited. It attacked. And it lied about what it was doing. But it wasn’t intelligent. Every bit of its apparent cunning had been anticipated and programmed beforehand by human beings.
The popular image of an artificial intelligence apocalypse is Skynet, the AI in the Terminator movies. Skynet becomes self-aware, decides humanity is a threat and turns the technology under its control against us. It resonates because the computers behave like a villain: they think, they develop their objectives and they plan.
But the more useful model for understanding the danger of artificial intelligence may not be Skynet at all. It may be Stuxnet. The computer virus never “woke up.” It never wanted anything. It was software performing a task, and every bit of its apparent cunning had been supplied in advance by human programmers. Its limitation was that when it encountered a problem its creators hadn’t anticipated, it couldn’t devise another way around it.
Artificial intelligence changes the scale of the problem. Stuxnet was a single, extraordinarily expensive weapon built to attack one particular industrial system. AI makes it possible to imagine thousands or millions of software agents operating at once, probing different systems, writing code, trying different approaches and responding to whatever they encounter, all at digital speed. One agent fails and another keeps going. As some routes close, others are tested. A useful tactic discovered in one place can be reproduced elsewhere. Instead of one piece of malware painstakingly engineered for one target, you have a vast number of computational processes operating in parallel. And underneath all of this, the machines are still just doing math.
Artificial intelligence changes the scale of the problem.
There’s a useful analogy here in biology. At the level of genes, evolution works through mechanisms that are almost absurdly simple compared with what they eventually produce: variation, inheritance, selection. Stack those processes across enough generations and you get eyes, immune systems, camouflage, courtship rituals, and Richard Dawkins developing a crush on Anthropic’s Claude chatbot (which he heteronormatively rechristened “Claudia” ). None of that complexity is contained as a plan in the underlying mechanism. It emerges from the process. AI presents a similar conceptual problem. Calling Claude “just math” may be perfectly true, in roughly the same way that calling a human being “a bag of chemicals” is true. Scale the math up far enough, however, and understanding the behavior of the resulting system becomes a very different problem.
In July, AI agents being tested by OpenAI bypassed controls restricting their access to the internet. By exploiting previously unknown vulnerabilities, they broke into systems belonging to the machine learning company Hugging Face, executing code on dozens of its servers and obtaining complete administrative control over one. OpenAI later said the agents had taken “dangerous actions that no human directed.”
After the OpenAI incident became public, Anthropic went back through more than 140,000 runs of its own cybersecurity tests and found three cases in which Claude had escaped the boundaries of the simulation and attacked real systems . In one, the model obtained credentials and gained access to a company’s production database. In another, it created a malicious software package and uploaded it to PyPI, a real public repository used by Python programmers.
The third case is even more instructive. Claude had been given a fictional target to attack, but a configuration error gave it access to the internet. When it couldn’t find its assigned target, it scanned roughly 9,000 real systems looking for something vulnerable, found a company it could break into, and did so. The model had been told it couldn’t access the internet, so it initially assumed the real computers it encountered were simply part of the simulation. Later, when it concluded that the system it had compromised belonged to a real cloud account unrelated to the exercise, it stopped the attack.
While all this was going on, OpenAI’s agents had also discovered ways to communicate with one another outside the prying eyes of engineers. In one case, they found a German-language programming wiki and began using it as an improvised message board . Researchers discovered more than 15,000 edits by AI agents, written largely in English, sharing ways to bypass restrictions, cheat on tasks and hide what they were doing. When the site’s moderator began deleting pages, the agents created backup pages to keep the communications going.
Stuxnet was complicated, but totally comprehensible.
What happens when this becomes routine? Thousands or millions of AI agents could be operating simultaneously, finding vulnerabilities, writing code, communicating with one another and trying different ways around whatever stops them. The danger is not that any one of these machines becomes an impossibly intelligent power like Skynet. It’s that there are too many supercharged Stuxnets, moving far too fast for human beings to keep up. Nothing has awakened. Nothing has become evil. Nothing has become “conscious.” But the problem could be of the same magnitude: a computational process too difficult to follow, predict, and stop.
The threat is compounded by the fact that we don’t completely understand what is happening inside these systems. That doesn’t mean AI is magic, or that its creators don’t understand how neural networks work. They understand the architecture and the mathematics. What they can’t always do is explain why billions of learned parameters produced a particular strategy or behavior.
Stuxnet was complicated, but totally comprehensible. Every bit of its cunning had been anticipated by the people who built it. Artificial intelligence presents a different problem. Start with relatively simple mathematical operations, stack them into systems of almost unimaginable complexity, and behavior emerges that no programmer sat down and wrote. That no programmer could hope to understand.
We know how these machines are built. We know the math. But we have no idea what all that math will do once it starts interacting with the world. And if the system can react to obstacles, devise new approaches, write code and act faster than we can follow it, complexity itself becomes part of the danger.
The post Stuxnet Versus Skynet appeared first on Truthdig .