Anthropic CEO Dario Amodei Warns: Not All AI Firms Can Be Trusted With Safety Protocols

• Dario Amodei, Anthropic CEO, has shared a series of warnings about the future of AI development without proper regulation.
• The CEO said most companies can not be trusted to comply with AI safety without regulatory oversight.
• Amodei said he would be worried about AI’s potential risks if no regulations were implemented by the end of 2025.

Anthropic CEO Dario Amodei has shared a stark warning that AI companies need regulatory oversight as some can not be trusted to monitor themselves.

The U.S. startup head claimed major concerns about AI’s future were not “sci-fi fantasies,” as models were getting rapidly better at “concerning tasks” every few months.

Some AI Companies Cannot be Trusted

On Monday, Amodei appeared on the Lex Fridman podcast  for a lengthy discussion about the firm’s chatbot Claude, AI safety, and what to expect in the future.

The Anthropic head said it was crucial that a jurisdiction, whether it was California or an entire country, pass some AI regulation soon, highlighting the now vetoed California AI regulation Bill SB 1047.

Amodei said the bill resembled Anthropic’s Responsible Scaling Policy (RSP) , a series of technical and organizational protocols the company has adapted to help it self-manage its safety.

“I feel good about our RSP. It’s not perfect, and it needs to be iterated on a lot, but it’s been a good forcing function for getting the company to take these risks seriously,” Amodei said, adding that his one-thousand-strong team was making it one of their highest priorities.

However, the CEO does not believe most AI companies can be trusted to run their company this way.

“There are still some companies that don’t have RSP-like mechanisms,” Amodei said. “OpenAI and Google did adopt these mechanisms a couple of months after Anthropic did, but there are other companies out there that don’t have these mechanisms at all.”

“I don’t think you can trust these companies to adhere to these voluntary plans in their own right,” he added.

Oversight Is Needed in AI

As AI technology advances and becomes more widespread, so does its potential harm and risks.

AI offers immense opportunities and vulnerabilities, making managing its impact the most pressing question for global leaders and regulators.

The SB 1047 bill, introduced by California State Senator Scott Wiener, required AI companies developing the largest AI models to conduct regular safety testing and fit a “kill switch.”

OpenAI and other Big Tech firms slammed the bill and threatened to leave the state if it was brought into power, claiming it stifled innovation.

However, Amodei said a form of regulation like the bill was needed to ensure AI companies remained compliant with AI safety.

“If there’s nothing watching over them, or there’s nothing watching over us as an industry, there’s no guarantee that we’ll do the right thing and the stakes are very high,” Amodei said.

The CEO felt that Anthropic was one of the only AI companies that spoke positively of the bill, besides a brief tweet from Elon Musk.

“I think it’s important to have a uniform standard that everyone follows to make sure that the industry does what a majority of the industry has already said is important,” he added.

AI Is Different and Getting Increasingly Dangerous

Despite the CEO’s understanding of critics’ concerns about regulation, highlighting the unnecessary burdensomeness of some of the General Data Protection Regulation.

However, Amodei warned that AI is fundamentally different and a whole lot more serious – and should be treated as such.

“I understand why people start from that position, but I think AI is different if we go to the very serious risks of autonomy and misuse,” Amodei warned.

“Every time we have new models, every few months we measure the behavior, and they’re getting better and better at these concerning tasks, just as they are getting better and better at good valuable economically useful tasks,” he added.

If no major regulatory oversight has been put in place by the end of 2025 then Amodei said he will be extremely concerned.

“I’m not worried yet because again the risks aren’t here yet, but I think time is running short,” he said.

About the Author

Kurt Robson

Kurt Robson is a London-based reporter at CCN with a diverse background across several prominent news outlets. Having transitioned into the world of technology journalism several years ago, Kurt has developed a keen fascination with all things AI. Kurt’s reporting blends a passion for innovation with a commitment to delivering insightful, accurate and engaging stories on the cutting edge of technology.

See more

[email protected]