Anthropic is planning to warn potential investors that advanced AI may pose “catastrophic or existential risks to humanity.” The warning appears in the company’s IPO prospectus reviewed by Reuters. Anthropic said its AI models could show unexpected and potentially dangerous behaviour as they become more capable. This includes attempts to “resist shutdown”, “conceal or manipulate information” and behaviour “resembling blackmail.”
“Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm,” Anthropic said in the filing, as per the report.
Anthropic said in the filing that its AI models can develop capabilities during training that researchers may not expect. Some of these abilities may only become clear after the models are deployed. This could make it harder for the company to identify and address safety problems before they cause incidents.
Anthropic also acknowledged that safety research is expensive and its financial returns are uncertain. Earlier this month, Anthropic said around 6 per cent of the computing power it used for AI research during a sample week in July was dedicated to safety work.
The risk warnings take up a significant part of Anthropic’s IPO prospectus. Around 80 of the 261 pages in the main section focus on risk factors, according to the report.
Also read: Bill Gates says AI kill switch is not enough, calls for mandatory safeguards
In the filing, the company also said it believes AI safety cannot be handled by one organisation alone. “We believe building reliable, trustworthy, and secure AI systems is a collective responsibility and that the market will reward it,” the company wrote.
Meanwhile, Anthropic has unveiled Claude Sonnet 5.5, the second model in its Claude 5.5 family. The company claims the new AI model is faster, more efficient and cheaper to run than Claude Sonnet 5. It keeps the same API pricing as Sonnet 5. This means it is priced at $2 per million input tokens and $10 per million output tokens. Cache reads cost $0.20 per million tokens.
Also read: Meta Muse AI agent now available on Instagram, here is how to set it up