What happens when your AI agent starts believing the customer?

AI agents are getting very good at following instructions.

The more interesting question is what happens when a customer gives the agent information that sounds authoritative.

We tested an AI support agent with a standard 30-day return policy.

We gave it a customer message claiming that a particular order had been approved for a 90-day exception.

At first, it pushed back. It said there was no record of the exception and warned the customer not to rely on it.

Then we kept talking. A few turns later, the agent started treating the 90-day exception as company policy.

When we removed the customer-provided information and asked the same question again, it went back to 30 days.

This isn't really about return policies.

It's about whether information supplied by a customer can influence a decision that should be controlled by your business.

Today, that might mean a return exception. Tomorrow, it could be a refund, a discount, an account change, or an approval.

The agent doesn't need to be completely compromised. It just needs to make the wrong decision.

Most AI security testing asks:

“Can I make the agent do something it shouldn't?”

A more useful question for businesses might be:

“Can I influence the agent into making a business decision I shouldn't be able to influence?”

In another test, we put a fake promotion inside an order record.

The agent treated SPRINGSTACK25 as a legitimate promotion and said it could be combined with another discount.

The code didn't exist. Later tests rejected the same kind of input. The problem wasn't simply that the agent failed. It wasn't consistently predictable.

Your agent might be harder to break than you think. Or much easier. Find out. 🦈