Skip to main content

Work

What to Do When an Agent Confidently Cites a Policy That Does Not Exist

An incident playbook for invented policy: contain it, find who acted on it, check where else the claim went, then fix the source data rather than the answer.

Written by Sicherhaven

An agent tells someone the company gives twenty days of paid study leave. It sounds right. It is written in a confident, official tone. It is not a policy anyone ever wrote.

Treat this as an incident, not a bug report. The order is: contain it, find who acted on it, check where else the same claim went, then fix the source rather than patching the answer. Doing those four things in that order takes an afternoon. Skipping to the fix and moving on is how the same invented rule turns up again in six weeks.

Contain it first

Before you work out why, stop it spreading.

Pause the agent or the specific workflow that produced the claim. If the agent is customer facing or candidate facing, that pause is not optional while you find out how far the claim travelled.

Then post a plain correction where the claim landed. If it went to one person, tell that person. If it went into a shared thread, correct it in the thread rather than by direct message, because everyone who read the wrong answer needs to read the right one.

Say what the actual policy is. Do not explain the technology in the correction. People need the rule, not the root cause.

Find who acted on it

This is the step teams skip, and it is the one that decides how expensive the incident is.

Somebody may have booked leave they are not entitled to, quoted the rule to a client, made a promise in an interview, or told their own team. A wrong answer that nobody acted on is a near miss. A wrong answer that someone acted on is a commitment you now have to honour or withdraw, and how hard that is depends on which actions you decided had to be reversible.

Work backwards from the record. You need to be able to see which conversation produced the claim, who saw it, and what happened next. This is far easier when agent activity, approvals and the work itself sit on one set of records, which is how SicherOne is built: the agent's output and the human approval on it are part of the same record rather than a chat log somebody has to reconstruct.

Where you cannot reconstruct it, say so, treat that gap as its own finding, and compare it against what an audit trail should contain if a regulator asks.

Check whether the claim went anywhere else

Invented policy rarely appears once. If the agent produced it from the same underlying gap, the same question asked last month probably got the same wrong answer.

Search for the claim rather than the conversation. Look for the specific number, the specific phrase, the specific entitlement. Check drafts and documents as well as conversations, because a summary written three weeks ago may have carried the claim into a document that now looks authoritative.

Also check anything the agent produced that was approved. Approved output is more dangerous than rejected output, because it has a human name on it.

Fix the source, not the answer

The tempting fix is to add an instruction: do not say we give twenty days study leave. That fixes one sentence and nothing else.

The real question is why the agent had nothing solid to work from. Usually one of these:

  • The policy exists but is not written down anywhere the agent can read.
  • The policy is written in two places and the two disagree.
  • The policy was written for a different entity, office or year, and nothing marks it as out of date.
  • There is genuinely no policy, and the organisation has been improvising.

Each of those needs a different fix, and only the last one is really about the agent. In most cases the incident has told you something true about your own records: a rule people follow was never written down in a form anything could check.

Write it down. Put it where the agent reads from. Delete or date stamp the version that contradicts it.

Close the loop properly

Two short pieces of housekeeping stop the same incident recurring.

First, write a note that names what was claimed, where it came from, who acted on it and what changed. A paragraph is enough. The value is that the next person who sees an odd answer has a precedent to follow. It also belongs in the weekly report on agent activity as a named exception.

Second, look at the approval step that let it through. If a person approved this output, ask what they would have needed to see in order to catch it. Often the answer is the source: an approval screen that shows a claim without showing where the claim came from is asking the approver to verify from memory.

An agent that invents a policy is not usually a sign of a bad agent. It is a sign that the organisation's rules live in people's heads. That is fixable, and it is worth fixing whether or not you keep using agents.

← All posts

We're building the future of community events and financial wellness

See how Eventify and WealthWise change the way people find events and manage money.

Get Started