OpenAI and Google DeepMind Staff Advocate for AI Whistleblower Rights as AI Safety Experts Quit in Droves
An open letter and subsequent resignations underscore a growing discontent within the AI community regarding the balance between commercial interests and safety priorities. Photo by Kent Nishimura / Getty Images.
Key Takeaways
- An open letter penned by Big Tech employees underscores a growing discontent within the AI community
- Employees working on OpenAI safety have been leaving the tech company over the last several months.
- Ilya Sutskever and Jan Leike – two high-profile figures who left OpenAI recently shared their concerns about OpenAI’s financial incentives.
- Since Sam Altman’s reinstatement five of the company’s most safety-focused employees have left.
A letter signed by eleven current and former OpenAI employees and two from Google DeepMind, calls for increased protections for whistleblowers and a more transparent approach to AI safety.
This initiative coincides with reports of numerous AI safety experts leaving OpenAI. The departure of these experts raises questions about the effectiveness and sincerity of OpenAI’s safety commitments.
Industry Leaders Warn of AI Risks and Lack of Oversight
A group of current and former employees from leading artificial intelligence companies, including OpenAI and Google DeepMind, have issued an open letter highlighting the urgent need for better safety oversight in the AI industry.
The signatories emphasize the importance of a “right to warn about artificial intelligence,” stressing that AI companies possess substantial non-public information about their systems’ capabilities and limitations. They argue that these companies have weak obligations to share safety-related information with governments and none with civil society, which raises significant concerns about their reliability in voluntarily disclosing potential risks.
OpenAI responded by defending its current safety practices, highlighting the existence of a tipline for reporting issues and ensuring that new technologies are only released after appropriate safeguards are in place. However, the lack of a response from Google further amplifies the concerns raised by the letter’s signatories.
A Need for Anonymity
The anonymity of some of the signees currently employed at AI companies like OpenAI and Google DeepMind, suggests that despite public assurances, the internal culture may not be as supportive of safety and ethical concerns as it should be.
In November, a letter purportedly sent to the OpenAI board by a group of former employees accused CEO Sam Altman of overseeing a “systematic silencing of dissent” that “created an environment of fear and intimidation.”
OpenAI Employees Call it Quits
The open letter follows the news of several prominent figures leaving OpenAI, citing concerns about the company’s focus on financial incentives over safety priorities. Ilya Sutskever, one of the co-founders, and Jan Leike, who led the alignment team, are among those who resigned in May. Their departures highlight a growing discontent within the organization regarding the balance between commercial interests and the imperative to ensure AI technologies are safe and beneficial.
Sharing their thoughts in an op-ed in the Economist , Sutskever and Leike say regulation is needed, adding that AI firms can’t be left to govern themselves.
The recent wave of resignations, particularly from employees deeply involved in AI safety, underscores the internal conflicts and raises questions about the company’s commitment to its stated safety goals.
The two high-profile exits are part of a larger exodus that began in November 2023, following a failed attempt by OpenAI’s board to fire CEO Sam Altman. Since then, approximately five additional safety-focused employees have either quit or been forced out.
Loss of Confidence in OpenAI?
The reasons for these departures go beyond immediate safety concerns about AI. Insiders suggest that the primary issue is a loss of faith in Altman’s leadership regarding safety commitments.
Critics have described Altman as prioritizing rapid AI development over safety, raising additional concerns among the safety team. His fundraising efforts, including seeking investments from regimes with dubious human rights records, have further alarmed employees dedicated to ethical AI use.
The super-alignment team is led by co-founder John Schulman. Schulman is heavily involved in maintaining the safety of existing products, leaving little room for proactive safety measures. This reduction in focus and resources jeopardizes the team’s ability to address future AI risks effectively, a concern previously highlighted by Leike in his calls for greater preparation and resources to manage upcoming AI models and their societal impacts.
This sentiment has been exacerbated by the company’s alleged practice of requiring departing employees to sign off-boarding agreements with non-disparagement clauses, often at the cost of their equity in the company. This policy, which potentially strips employees of significant financial assets if they refuse to comply, has further fueled distrust.
Is the AI Safety Pledge Enough?
The recent pledge by major AI companies, including OpenAI, at the Seoul AI Safety Summit to implement a “kill switch” involves developing mechanisms to instantly shut down AI systems if they exhibit harmful or unintended behaviors. The Frontier AI Commitments goal is to enhance safety and prevent misuse of advanced AI technologies.
The AI pledge includes other safety measures such as watermarking AI-generated content to combat misinformation and testing AI tools for security by independent experts before public release. These voluntary commitments aim to set a precedent for global AI safety standards and foster collaboration between tech companies and governments to mitigate the risks associated with AI.
While the “kill switch” pledge is a positive development for AI safety, its impact will depend on how well it is implemented and whether OpenAI can retain and leverage the expertise needed to make these safety measures effective.
The loss of key personnel with expertise in AI safety over the past several months speaks broadly of the concerns surrounding AI technology. These experts’ departure might indicate deeper issues within the organization regarding its approach to safety and governance.
Jan Leike’s Resignation Damning of OpenAI’s “Core Priorities” and Safety Culture
