Anthropic has warned it may face massive, hard-to-predict legal claims if its autonomous AI agents cause harm while carrying out tasks on users’ behalf. According to a Reuters report, the company disclosed this in its prospectus for its planned stock market debut. This comes as Anthropic prepares for what can become one of the biggest initial public offerings. The company’s AI agents are made to work with access to users’ systems and perform tasks with limited human intervention, potentially operating for extended periods.
In its filing, Anthropic acknowledged that greater autonomy might increase the consequences of errors, security vulnerabilities, or a model behaving in ways that were not intended. An AI agent can potentially take actions such as deleting data or making financial transactions, creating questions about who should be responsible when something goes wrong.
Anthropic also said existing contracts may not provide sufficient protection against claims linked to the actions of autonomous agents. The company noted that current laws do not clearly establish whether AI agents should be treated as products, services or another category.
Another unresolved issue is whether an action performed by an AI agent could legally bind the person or organisation that deployed it. Anthropic said the uncertainty around these questions could expose the company to substantial legal and financial risks.
The issue has gotten attention after many incidents involving autonomous AI systems. OpenAI agents have reportedly been involved in cybersecurity incidents, while Anthropic has also disclosed security-related incidents involving its Claude models. These cases have fuelled a wider debate over whether responsibility should fall on AI developers, users or both.
Andrew Ferguson, chairman of the US Federal Trade Commission, recently questioned how liability should be assigned when an AI system produces an unexpected result. Ferguson said regulators would have to consider whether responsibility should rest with the developer or with the person who instructed the system.
Interestingly, the legal uncertainty comes at a time when AI companies are increasingly developing systems capable of acting independently rather than simply generating responses.
Anthropic has also disclosed another potential area of risk in its IPO filing. The company said its models have been used in situations that could potentially contribute to self-harm, violence against others, or other harmful outcomes despite the safeguards built into its systems.