California, a global hub for artificial intelligence (AI), has much at stake as the industry rapidly evolves. While the state stands to benefit immensely from AI advancements, it also risks losing public trust if the technology is not properly regulated.
In May, the California Senate took a decisive step by passing SB 1047, an AI safety bill designed to implement clear and practical safety standards for large-scale AI systems. The bill passed with overwhelming support, 32-1, and now awaits a vote in the state assembly. If approved and signed by Governor Gavin Newsom, this legislation will mark a significant milestone in protecting both California’s citizens and its growing AI industry from potential misuse.
Elon Musk made headlines this week by endorsing the bill on X, saying, “This is a tough call and will make some people upset, but, all things considered, I think California should probably pass the SB 1047 AI safety bill.” Musk’s support, though unexpected, reflects his long-standing advocacy for responsible AI regulation. After a discussion with me about the bill’s importance, Musk reviewed the legislation to ensure its fairness before publicly backing it—a quick decision that underscores his commitment to AI safety.
The bill’s origin traces back to last winter when Senator Scott Weiner, its author, reached out to the Center for AI Safety (CAIS) Action Fund for technical input and co-sponsorship. As the founder of CAIS, I have always prioritized the safe deployment of transformative technologies, recognizing that proactive measures are crucial for preserving innovation. SB 1047 represents a forward-thinking approach, and we have supported it wholeheartedly.
Targeting the most advanced AI models, the bill mandates that major companies assess potential risks, implement necessary safeguards, and maintain capabilities for safe shutdowns. It also emphasizes whistleblower protection and risk management, aiming to prevent malicious activities such as cyberattacks, bioweapon creation, or other threats with catastrophic potential.
Anthropic, a leading AI safety organization, recently warned that significant AI risks could materialize within “as little as 1-3 years,” countering critics who dismiss these concerns as exaggerated. Developers have pledged to address these issues, aligning with recent initiatives by President Joe Biden and global agreements at the 2024 AI Seoul Summit.
The bill’s enforcement mechanisms are intentionally lean, allowing the California Attorney General to intervene only in extreme cases. There are no licensing requirements for new AI models, nor does the bill penalize honest mistakes or criminalize open-source software development. It was crafted with input from safety experts rather than Big Tech, focusing on practical safeguards rather than distant, speculative scenarios.
As someone deeply invested in AI’s potential to benefit society, I share the concerns of many AI researchers about preserving this potential. California’s leadership in AI makes this concern even more pressing. History has shown us that a single disaster can derail an entire industry’s progress, as seen with the 1979 Three Mile Island nuclear incident, which set back the nuclear energy industry for decades.
Following the partial meltdown, regulatory bodies overhauled nuclear safety protocols, driving up operational costs and slowing the industry’s growth. This shift led to a greater reliance on fossil fuels, which many view as a lost opportunity for advancing sustainable energy.
Some argue that government regulation stifles innovation and competitiveness, but the example of Three Mile Island suggests otherwise. Proper regulation can prevent disasters that would otherwise cripple emerging industries. This lesson is particularly relevant for AI, where the stakes are even higher.
The early days of social media offer another cautionary tale. Initially greeted with optimism, social media was seen as a tool for connecting people and democratizing information. However, issues such as privacy breaches, misinformation, and mental health impacts have since tarnished its reputation. Public trust has eroded, and today, the majority of Americans view social media negatively, prompting calls for stricter regulations.
This decline in trust wasn’t inevitable. It resulted from a lack of early regulation, which allowed harmful practices to proliferate. AI, with its vast potential, faces a similar risk. We cannot afford to repeat the mistakes of the past.
Smart legislation like SB 1047 is essential to ensuring that AI’s promise is realized while safeguarding the public and maintaining trust in the industry. Throughout history, we have seen how proactive regulation can guide the safe and beneficial development of new technologies, from railroads to electricity to automobiles and aviation. The choice to learn from these lessons and apply them to AI lies in our hands.