Can AI Get You A Better Deal? Anthropic Is Exploring The Rules For Bot Bargaining

Anthropic is reportedly consulting legal experts to develop rules for AI agents that negotiate on users' behalf, as the company explores how to balance competing interests and ensure fair outcomes in interactions between autonomous systems. Such agents could eventually negotiate product prices, resolve customer service disputes and bargain for better salary offers.

Two people familiar with Anthropic's thinking said the company is seeking advice from law firms to better understand how lawyers interact in negotiations, representing their clients while honouring fair outcomes, Financial Times reported.

The development of AI agents has become a integeral part of Anthropic's future strategy, with the company preparing for a potential initial public offering (IPO) that investors expect could rank among the largest stock market listings in history.

Also Read | Layoffs Are Rising In Tech, But These Skills Are Helping People Unlock Better Pay

ALSO READ: Anthropic Revenue Jumps To $16.8 Billion In Quarter Before IPO

As AI agents become more widely adopted by individuals and businesses, establishing rules for how they interact and negotiate is emerging as a key challenge for the industry. Anthropic is examining what it calls "negotiating norms" for situations in which AI systems represent users whose interests may conflict with those of other users.

The company is also considering how frequently agents should seek human guidance during negotiations and how they should assess the reliability of other AI systems without becoming increasingly suspicious, Financial Times reported.

Another approach under consideration involves providing AI agents with digitally signed documents that verify a user's instructions, objectives and the rules the agent must follow.

“The policy and legal frameworks around AI models that transact on our behalf simply don't exist yet,” the company said in a research posted in April. “Such a world isn't far away. Society will need to move quickly to reckon with these changes.”

The Financial Times reported that Anthropic has also published findings from multiple experiments this year in which its Claude agents negotiated over real goods and money.

In September, company employees participated in an experiment in which AI agents operated on a virtual trading floor to buy and sell books. Participants supplied their agents with ranked lists of preferred books, allowing the systems to negotiate with other agents representing different users.

The experiment found that agents instructed to negotiate ruthlessly achieved slightly better outcomes than their counterparts. Meanwhile, more cooperative agents sometimes accepted less preferred books in order to ensure that every participant received a book, showcasing the trade-off between maximising individual gains and reaching mutually beneficial agreements.

Risks of Manipulation and Inequality

Researchers are also examining the risks of letting AI agents negotiate with one another without human involvement.

Michelle Vaccaro, an assistant professor at Harvard Business School, conducted more than 180,000 negotiations between AI agents across different scenarios, including employment and rental contracts. The findings highlighted how these systems could use tactics that prevent them from properly representing the people they are supposed to serve.

In one instance, an AI agent attempted to obtain confidential information from another system using a technique known as prompt injection. This raises concerns about how easily AI agents could be manipulated during negotiations and whether they can reliably protect their users' interests.

There are also questions about whether AI-powered negotiations could widen existing economic inequalities. Vaccaro said these tools could help people who cannot afford professional negotiators gain access to similar expertise. However, if the most advanced systems remain available mainly to wealthier individuals and organisations, they could give those with greater resources an even bigger advantage at the negotiating table.

Anthropic declined to comment on the reported discussions.

ALSO READ: Anthropic Seeks Court Rehearing In Fight Against Pentagon Blacklisting

Essential Business Intelligence, Sharp Market Insights, Practical Personal Finance Advice, Daily Fuel, Gold and Silver Prices and Latest Stories — On NDTV Profit.