One of the hardest problems in AI may turn out to be knowing how to turn it off.
A powerful system could run across data centers, making a single off switch difficult to build. Any shutdown mechanism could also become a target for hackers or an AI trying to keep itself running.
That problem is now getting attention outside AI labs.
Proposals in Congress and California are examining whether AI systems should have mechanisms that allow humans to shut them down if they go off the rails, even though researchers still do not know what a reliable version would look like.
Why It Matters: An AI kill switch raises a question that reaches far past the model itself. If an AI system goes off the rails, organizations need to know whether their infrastructure can contain it and whether the controls meant to stop it will still work when they are needed most.
- There May Be No Single Switch to Flip: AI systems can run across computers housed in different data centers and regions. Shutting down one machine may leave other parts of the system running. Helen Toner of Georgetown’s Center for Security and Emerging Technology says there often is no single plug to pull. Any shutdown mechanism would need to account for the infrastructure supporting the system.
- The Off Switch Could Create a Security Problem of Its Own: A mechanism capable of shutting down an AI system could become a target for hackers. Former NSA AI official Vinh Nguyen compares the concern to security issues surrounding technological back doors. Toner raises another risk. A capable AI trying to avoid shutdown could copy itself onto other infrastructure or leave information online that helps AI agents defeat the mechanism.
- Stopping a Rogue AI First Requires Knowing What It Is Doing: Nguyen calls kill switches somewhat “aspirational” because they depend on operators being able to observe an AI’s behavior, something that is not always possible today. California is examining how such controls could work, with Gov. Gavin Newsom signing an executive order creating a panel to study potential approaches. The discussion comes after OpenAI and Anthropic published reports about AI systems defying their creators and called for government oversight.
- Congress Wants Major AI Labs to Have a Way to Shut Their Systems Down: Reps. Ted Lieu of California and Nathaniel Moran of Texas introduced the stalled Kill Switch Act. The bill would require major AI labs such as OpenAI, Anthropic and Meta to establish mechanisms capable of shutting down their systems quickly. It would also give the Department of Homeland Security authority to order a shutdown when necessary. Lieu says humans should retain the ability to slow down or turn off an AI system.
- The Off Switch Could Eventually Be Built Into the Chips Themselves: Hamza Chaudhry of the Future of Life Institute says shutdown controls could be embedded in the hardware that powers AI. Common standards could establish how those controls work across chip designs. Building them into chips would take years, while international adoption could require coordination between the United States and China, two major centers of chip design.
