Smart AI agents need smarter guardrails
Artificial intelligence technology is evolving faster than many consumers, or even small companies, can fully understand the risks created by its new capabilities. Agentic AI is one such milestone in this evolution.
Unlike generative AI, which primarily responds to user queries, drafts documents or synthesizes information, agentic AI can take action on behalf of humans.
Users spell out the goal to the AI agent, and the system breaks it down into smaller steps, engages with websites, fills out forms, makes bookings, purchases items, uses external tools and even coordinates with other agents to achieve the objective.
Think of a future where one person or one company deploys hundreds of specialized agents to complete different tasks.
But agents, just like chatbots, suffer from the limitations of generative AI. They are non-deterministic systems and can hallucinate. The same question may produce different answers at different times. They are also prone to adversarial attacks.
The bigger risk is that agentic AI does not merely generate information, but can also take unintended actions. One agent's error can snowball into a bigger problem with downstream agents' actions.
Since AI agents operate across systems, products and borders, they will certainly add more complexity to cyberspace and digital commerce. Soon, your shopping agent might negotiate with a merchant's pricing agent over the price of an air ticket or a household finance agent might dispute a bill with a utility company's chatbot. With the increased complexity of multi-agent systems, strong safeguards are required to protect consumers.
The key question is: If AI agents can make decisions on behalf of humans, where should we draw the line? How can we ensure that agents always act in the best interests of the consumers who deploy them, and can resist manipulation? These are no longer hypothetical questions.
The first priority is safety. When a conventional chatbot gives bad advice, the damage is limited to wasted time.
But agents capable of taking action can turn bad judgment into real-world consequences at machine speed — booking travel incorrectly, transferring money, signing up for unwanted subscriptions, negotiating with other agents, or purchasing expensive products.
An agent that misinterprets a contract clause can be manipulated into accepting an unfavorable deal by a more persuasive counterpart.
Safety in this context means more than avoiding useless answers. It means establishing clear boundaries of authority.
Consumers need agents that operate within adjustable limits: monetary caps on purchases, categories of decisions that always require explicit human approval, and safeguards to prevent irreversible actions. An agent should never be able to do something a reasonable person would not let a new employee do without confirmation.
Therefore, we need safe protocols for identity management, and authorization mechanisms that can work across different products and platforms.
Reliability is the second pillar, and it is trickier than it sounds. Reliability is not just about uptime or accuracy. It means that an agent consistently understands and follows the intent of the user.
An agent that misinterprets a user's intent, or can be manipulated to misunderstand, is counter-productive. Multi-agent commerce creates an environment where agents constantly communicate with other agents, some of which may be designed by companies with an obvious interest in extracting more data, money or attention from the consumer.
A retailer's agent may be optimized to upsell; a booking agent may be designed to nudge users toward options that pay higher commissions. To protect consumers, we need traceability and transparency.
Users should be able to understand what steps their agents take and why. We need transparency between agents from different vendors to ensure they are not colluding.
Even when the authorized agents work properly, agentic AI is prone to malicious attacks to manipulate its actions. If a consumer's agent is not secure and robustly built to resist these tactics, the whole premise of "delegating decisions to save time" collapses into "delegating decisions to be quietly exploited". Attackers can extract consumers' private information, direct them to malicious websites or influence agents into making harmful choices.
Reliability therefore requires agents that are auditable and robust. Just like banks regularly test their systems against fraud, AI agents must be stress-tested against the exact kinds of persuasion and manipulation techniques they may encounter from other commercial agents or malicious actors.
Finally, there is the question of liability. We need clear rules for who is responsible when an agent causes harm: the company that built the underlying model, the user who deployed the agent, developers of the protocols connecting the merchants, or the websites with manipulative agents.
Ambiguity here does not benefit the consumer whose interest is harmed. A sensible framework would hold the deploying party primarily responsible for the actions their agent takes, while requiring model and platform providers to meet baseline safety and transparency standards.
None of this means banning agentic AI or treating it as inherently untrustworthy. Agents could in fact lead to significant savings in time and money. They could lower entry barriers or costs for entrepreneurs.
But the technology's usefulness depends entirely on trust, and trust depends on rules that exist before the harm occurs, not after it.
We need to establish clear guardrails and clear chains of responsibility. Any tool this powerful needs safeguards proportional to what it can do or enable.
The author is the founder of AIethicist.org, president of the Center for AI & Digital Policy and the recipient of Women in AI Awards North America 2023.
The views don't necessarily reflect those of China Daily.
If you have a specific expertise, or would like to share your thought about our stories, then send us your writings at opinion@chinadaily.com.cn, and comment@chinadaily.com.cn.
