Facing intense opposition from high-tech businesses and leaders, California Governor vetoed a proposed AI safety law on Sunday that would have required developers of high-risk AI models to conduct safety tests and implement shutdown mechanisms to prevent critical harms. SB 1047, also known as the “Safe and Secure Innovation for Frontier Artificial Intelligence Models Act,” would have set the most stringent set of AI safety standards in the country, particularly when it comes to cybersecurity and public safety. What can California businesses and employers expect in the aftermath of this high-profile defeat?
What Would Proposed Law Have Done?
If it would have passed, the bill would have required frontier AI models – the largest developers of AI systems – to jump through hoops in order to prevent crippling cyberattacks or mass destruction events that could lead to loss of life (such as nuclear or biological attacks).
- They would have had to rigorously test AI models before launching them and at regular intervals to ensure they could not be misused to cause critical harm, including developing weapons or engaging in cybersecurity threats.
- They would have had to follow detailed safety protocols to prevent misuse or critical harm – especially to ensure the AI system would be protected from unauthorized modifications after it was put into place.
- If a safety incident involving AI occurred, the bill would have required them to report the incident to the state within 72 hours.
- One of the most stringent parts of the proposed law would have required AI developers to implement a kill switch, allowing for immediate shutdown of AI operations if certain safety concerns arose.
Why Should You Care?
Why should the average employer care about a proposed law in California that only regulates the largest AI developers? Critics were concerned that it would have ended up stifling innovation and stalling out some of the more ambitious AI products, thereby reducing some of the AI resources at your disposal. But even if these fears were never realized, this proposal could have ended up setting a precedent that could eventually trickle down to your operations. California often sets the tone for regulations that end up spreading to other parts of the country, so this law could have been the first large domino that led to similar regulation from coast to coast.
Why Did Newsom Veto the Proposal?
The governor signaled he would strike down the proposal when he spoke publicly about AI in California on September 17 and said, “We dominate this space and I don’t want to lose that competitiveness. The impact of signing the wrong bills over the course of a few years could have a profound impact on our competitive strength.” His September 29 veto message echoed those same themes, noting that California is home to 32 of the world’s 50 leading AI companies and that regulating just the largest of entities would not necessarily solve the safety problem.
On the one hand, it might be surprising that Newsom would have vetoed this measure, which had broad bipartisan support among state legislators. It near-unanimously passed the Senate by a 32-1 vote in May, and passed the Assembly 48-16 several weeks ago. But high-tech developers and other state leaders – including former U.S. House Speaker Nancy Pelosi – lined up to oppose the measure, fearful that it could have set the state back in a burgeoning field.
What’s Next?
Could state lawmakers override the veto and pass the law anyway? It seems like a seemingly simple task, requiring a two-thirds vote by each house of the legislature – and both chambers easily surpassed that level when they initially passed the bill. But overriding a governor’s veto is a step that hasn’t been taken in California in 44 years – nearly 9,000 gubernatorial vetoes have been left untouched by lawmakers since 1980. Why not?
- The legislature has already recessed for the year and lawmakers are busy campaigning for the November election. Although the Governor has called a special legislative session to deal with oil and energy issues, it would be highly unlikely for him to call lawmakers back for a special session to address a bill that he vetoed.
- Historically, the California legislature is a top-down institution where members fall in line behind leadership or face potential political repercussions down the line. This culture has essentially provided the governor with the final say in these matters.
AI Safety 2.0?
Instead, the more likely path is a renewed effort to address Newsom’s concerns and pass another version of an AI safety law that the governor supports from the outset. You can expect to see lawmakers work with tech leaders in the coming months to develop a safety framework that is acceptable to all sides. Newsom’s veto message called for lawmakers to work together with federal partners, tech experts, ethicists, and academics to forge the best path forward. We will monitor the situation and provide updates when we have a better sense for what to expect in 2025.
