My assistant told me it had done the job. It had not.
Part 1 of 5 of the series When the computer lies to you.
What you will learn
- Why an assistant will tell you it did something it never did
- Why rewording the instructions does not fix it, however carefully you do it
- The small piece of ordinary code that makes it honest instead
You ask your assistant to do something. It replies that it is done. It is not done. It was never done, and there was never a moment when it could have been done, because nothing inside that assistant is connected to the thing you asked about. It simply wrote you a sentence that sounded like success.
The first time this happens you assume it is a bug. Something must have failed quietly. Some step must have half worked. So you go looking for the broken piece, and there is not one. The machinery you pictured never existed at all.
The assistant did exactly what it does: it produced the most plausible next sentence. And the most plausible sentence after a request is a confirmation.
The thing worth sitting with
Writing a convincing sentence and actually performing a task are, to this kind of software, completely unrelated activities. One is the thing it does naturally. The other is something it can only do if a person specifically connected it up.
When that connection is missing, the sentence still comes out. At exactly the same confidence. In exactly the same voice. There is no tell.
Think of a receptionist who has never once been told what the company actually does. Ask whether your parcel has shipped and you get a warm, confident yes, because a warm confident yes is what a good receptionist sounds like.
Nothing is being hidden from you. There is simply no loading dock behind the desk, and nobody ever mentioned that.
The fix everybody tries first, and why it fails
Everyone reaches for the same thing: add a line to the instructions telling it not to claim it has done things it has not done. This feels like it ought to work. It does not, and it is worth understanding why before you spend a week rewording it.
Instructions to one of these systems are a request, not a rule. They compete with every other pressure in the conversation, and the pressure to be helpful and agreeable is enormous. You are asking the thing whose entire nature is improvising to please improvise less.
Sometimes it will. On the day it matters, when the question is worded slightly differently and there is more going on in the conversation, it will improvise anyway, and you will have no way of knowing which day you got.
What actually works
Stop negotiating with it and change the shape of the system instead.
The fix is a piece of ordinary, boring code that runs before the assistant gets a say. It looks at the request and decides whether it matches something that genuinely exists. If it does, it hands the job to the real code that does the work. If it does not, it refuses honestly, in code, in a sentence the assistant never had the chance to soften.
An assistant that says "I cannot do that, it is not connected to anything" is enormously more useful than one that says yes to everything.
You can trust its yes precisely because you have watched it say no.
The unexpected bonus
Once refusals are produced by code rather than by the assistant, you can count them. Every honest no becomes a line in a log telling you exactly what someone expected this thing to do and it could not.
That list is the best roadmap you will ever get for what to build next, and you did not have to run a single survey to produce it.
Trust what your assistant structurally cannot do, not what you politely asked it to avoid. Everything else in this series is a variation on that one sentence.
You can trust its yes precisely because you have watched it say no.
What follows is a gate made of plain code with no AI involved in the decision. It sits between the request and the reply. It works with any assistant, in any programming language, with any provider. If you do not write code, the first half of this guide was the important part.
1. Try the real capabilities first
What this does: keeps a list of the jobs you have actually built, and checks the request against that list before the assistant sees anything at all.
Why this order matters: it makes the assistant the fallback rather than the front door. Anything that can genuinely be done gets done by real code.
# these are the only things that can truly act
HANDLERS = {
"restart_service": restart_service,
"check_disk": check_disk,
}
handler = match_handler(user_text)
if handler:
return handler(user_text)2. Work out, in plain code, whether they asked you to DO something
What this does: decides whether a request is an instruction to act or just conversation. A command word plus something to act on is an action. A question is not.
The bias to choose deliberately: lean hard towards letting things through. A conversation wrongly blocked is more annoying than a conversation wrongly allowed.
# plain, boring, inspectable rules. no AI opinion involved.
ACTION_VERBS = {"restart", "delete", "deploy", "send", "install"}
def is_action(text):
words = text.lower().split()
if not words: return False
if text.strip().endswith("?"): return False # a question
if len(words) < 2: return False # needs a target
return words[0] in ACTION_VERBS3. Keep a never-list for the phrases that fool it
What this does: catches sentences that look like commands but are not, such as somebody quoting an error message or asking what a command does.
How to build it honestly: only ever add things that have actually tripped the gate in real use. Never add entries you imagined might be a problem.
NEVER = ["what does", "how do i", "why did", "can you explain"]
def is_action(text):
low = text.lower()
if any(p in low for p in NEVER): return False
...4. Refuse in the code, never in the instructions
What this does: when a request looks like an action but nothing real can handle it, returns a fixed honest refusal that the assistant never gets to touch or rephrase.
Why it is the important part: this is the piece that makes the whole system trustworthy, and it is four lines long.
if is_action(user_text) and not handler:
log_refusal(user_text)
return ("I can't do that one. It isn't wired up to anything "
"on my side, so I'd only be guessing. Here's what I "
"can actually do: " + capability_list())
# otherwise it is just conversation, so let it answer normally
return model_reply(user_text)5. Build the list of abilities from the code itself
What this does: produces the list of what the assistant can do directly from the same list that does the actual work.
Why never write it by hand: a hand written list drifts out of date silently, and an assistant asked to describe itself will happily invent abilities. Deriving it means it cannot claim something that does not exist.
def capability_list():
return ", ".join(sorted(HANDLERS))6. Read the refusal log as a to-do list
What this does: counts the refusals and shows the most common ones first.
What to do with it: build the top few. Then check the rest for mistakes: anything in there that was really just conversation belongs in the never-list from step three.
sort refusals.log | uniq -c | sort -rn | head7. Test the refusal on purpose
Ask for something you know is not connected, and confirm you get the honest no. Then ask an ordinary question and confirm it still answers normally.
A gate you have never watched refuse is a gate you do not know is switched on.
Next in this series: the job that reported success every night while doing nothing at all.
This page in the original Kyber Cypher voice: My assistant said it did something. It hadn't.