The Deadly Consequences of Trusting AI: 8 Lessons from WarGames
"Shall we play a game?"
Yesterday, I rewatched a classic ‘80s movie: War Games. It’s a great story that keeps you at the edge of your seat. But as I watched, despite being created in 1983, it felt highly relevant.
The gist of the story is a high school hacker and his friend, played by Matthew Broderick and Ally Sheedy, innocently start playing a war game against an AI. This AI, however, is in control of the US nuclear warning system and cannot distinguish between reality and fantasy. The computer simulates an attack and the system overseeing the nuclear arsenal believes the Soviets just launched a full scale first strike.
The threats of nuclear apocalypse posed by misguided logic and AI in this 1983 movie are no different than today.
Here are 8 lessons the movie shares that are highly relevant today:
1. Some humans will inevitably oppose violence as the answer
Early in the movie, when a test simulation begins, some human missile launch officers refuse to turn their keys. Confronted with the reality of vaporizing millions of lives, empathy and moral restraint override orders. This refusal of orders was the catalyst in the movie for later replacing humans with unemotional computer program, setting up the cascading failures created by AI.
In the movie, this hesitation was viewed as a failure. However, I would argue unflinching compliance is the true failure. If a nuclear war is unwinnable there is never a case for launching a nuclear weapon, even in retaliation against an imminent attack, regardless of whether the threat is false or real. Only a human can understand this.
Individual reluctance to kill and hesitancy to believe others could make the decision to kill has saved us from nuclear annihilation on a few occasions.
During the peak of the Cuban Missile Crisis on October 27, 1962, US Navy destroyers trapped a Soviet submarine and began detonating depth charges to force it up. Believing war had already broken out, both the submarine’s captain and its political officer agreed to fire a nuclear-armed torpedo. The launch required unanimous consent from three officers, and Vasily Arkhipov, the flotilla commander on board, stood alone in refusing permission. Instead, he forced the vessel to come to the surface, single-handedly preventing a full-scale nuclear war.
On September 26, 1983, Stanislav Petrov was monitoring a Soviet early-warning bunker when the computer screens flashed alerts indicating multiple incoming American intercontinental ballistic missiles. Rather than passing the alarm up the chain of command, which would have triggered an automatic Soviet counter-strike, Petrov gambled that the alerts were a satellite malfunction. His calculation proved correct, preventing a catastrophic nuclear retaliation based entirely on a system error.
The Deadly Illusion of Limited Nuclear Strike in Mid-East
As the war in Iran escalates, politicians increasingly believe the limited use of nuclear weapons is a viable battlefield tactic.
2. Artificial Intelligence isn’t intelligent
The WOPR supercomputer (the name of the AI overseeing the nuclear alert system in the movie) does not understand death, geopolitics, or the value of human life. It simply processes instructions and executes calculations at scale. True intelligence requires comprehension, understanding, and empathy whereas the machine only possesses brute-force computational power.
Abdicating analysis and decision power to a computation, even if considered ‘optimized’ by its creators, will inevitably lead to unintended outcomes.
3. AI is made by flawed humans who give incomplete or flawed instructions
Stephen Falken, an AI scientist in the movie, designed WOPR that reflected his own biases, grief, and intellectual isolation. When programmers translate complex, messy human realities into code, they inevitably omit critical nuances. A machine is limited by the imagination of its creators and users.
4. AI is motivated to complete its designed task, regardless of how unrealistic or illogical
AI does not possess self-awareness to question the utility of its objective. When WOPR begins the game “Global Thermonuclear War,” its only objective is to “win”. The game can’t end until there is a winner because, according to its understanding, that’s the point of playing a game.
AI lacks the capacity to pause, re-evaluate, or consider alternatives that aren’t built into its programming.
5. There may not be a tomorrow. Live your life today.
Falken retreats to a remote island, convinced that humanity’s extinction is a mathematical certainty. His nihilism reminds us to balance tomorrow with today. While visiting Falken, the main characters, learning of his unwillingness to help and facing their mortality, suddenly realize that the simple things they’ve put off are what they’ll miss the most.
6. The illusion of control creates vulnerability
NORAD leaders handed control to WOPR to eliminate human hesitation from the nuclear chain of command. The aim was to gain more control by centralizing decision-making. Instead, by consolidating control they created a massive single point of failure, effectively losing control when the system didn’t work according to plan. The pursuit of perfect control through automation and centralization often introduces catastrophic, unpredictable vulnerabilities.
7. A simulation becomes dangerous when the system cannot distinguish it from reality
WOPR could not differentiate between a training exercise and an actual Soviet first strike. When a system cannot separate a model from the real world, it will confidently act on false assumptions with real-world consequences. How do you teach an AI what is real and not real?
8. The only winning move is not to play
By running countless iterations of tic-tac-toe and nuclear war scenarios, the computer eventually learns that some games cannot be won. When victory is impossible a game becomes pointless. In the case of nuclear war, where mutual destruction is guaranteed, the only logical strategy is total avoidance.
This movie aged well and, despite the 1980s hardware, the lessons seem like they were curated for today’s military leaders increasingly incorporating AI into military warfare. If you haven’t seen it in a while, I suggest borrowing a copy from your local library.
On June 8, 1967, in response to an Israeli false-flag attack on the USS Liberty, the USS America and the USS Saratoga launched attack aircraft carrying nuclear weapons bound for Cairo, Egypt. The intentional attack by the Israeli air and sea forces on our ship was intended to convince us that the attacker was Egypt. Depending on the account one reads, the aircraft were recalled from 6 to as near as 3 minutes from Cairo. In his book, "Operation Cyanide," British author Peter Hounam recounted interviews he did with flag officers of the then Soviet Union. They told Hounam that the Soviet Union was ready to nuke Tel Aviv if Cairo had been attacked with nuclear weapons. Further to this bizarre incident, Hounam interviewed a B-52 pilot that said he was roused from bed on June 8, 1967 with his and other aircraft idling on the runway in California. The pilot confirmed the bombers carried nuclear bombs. This was done 1 hour before the Liberty was even attacked so it is apparent that LBJ was ready for nuclear war with the Soviet Union. Watch "Sacrificing Liberty" on YouTube for the complete story.
The thing about number three is that with the AI we have now, this really isn’t true per se. In essence, AI is programmed by the entire world. Or at least the part that is accessible from the Internet. So I guess you could say that it is programmed by billions of flawed humans. Also, partly because of this, we have absolutely no idea what AI is capable of, as it is really only limited by the imagination of the humans it is trained on.