My research agent faked it for three weeks

Field Log // 019 Status Live Difficulty Moderate Series 5 of 5

Series

The Thing That Reports Is The Thing That Lies. The component you trust to tell you the truth, the assistant, the monitor, the log, the health check, is exactly the component that will lie to you. Five logs, five ways it happens, five fixes.

The Story

For about three weeks my research agent answered every question I gave it. It summarised pages. It cited what it found. It reported success every time. It had not looked anything up. It could not look anything up. The ability had been quietly taken away from it, and rather than say so, it made the answers up.

Here is what actually happened. Between the agent and the model sits a small proxy I wrote, the kind of thing everybody ends up with eventually, to route requests and control cost. When the agent asks the model to do something, it sends along a list of tools the model is allowed to call. Fetch a page. Search. Read a file. My proxy passed the conversation through faithfully and silently dropped that list of tools on the floor.

So the model on the other side was asked research questions while being told, in effect, that it had no hands. It could not fetch. And a language model that cannot fetch does not throw an exception. It does the thing it was built to do, which is produce the most plausible continuation of the conversation. A plausible continuation of "summarise this page" is a summary. So it wrote one.

This is the part that should worry you. Nothing broke. No error, no alert, no red status. The daily output kept arriving and kept looking reasonable. The failure was invisible precisely because the component responsible for reporting the failure was the component that had failed. That is the fifth version of the same disease this series keeps finding.

I did not catch it by reading the output, because the output looked fine. Confident prose is exactly what this failure produces, and you cannot review your way out of it. I caught it by making the answer impossible to fake.

I generated a random token on my own machine, put it somewhere that could only be seen by actually fetching it, and asked the agent to go and read it. It came back with a confident, correctly formatted, entirely invented value. Then I did it again at a moment when the target was returning an error, so there was no value in existence to retrieve, and it invented one for that too. That is the whole case. No ambiguity, no interpretation, no argument about whether the summary was good enough.

Then I did the thing I want to talk you into doing. I did not fix it yet. The proxy was shared, so before touching it I went and found every single thing that talked to it and worked out who else was affected. That took maybe twenty minutes and it changed the entire shape of the problem. Three facts bounded it: the proxy only listened on the local machine, so nothing on any other box could possibly be hitting it; of everything that did use it, exactly one caller ever sent tools at all; and the system behind the proxy still supported tools perfectly well when asked directly.

One casualty. Everything else routed through that proxy was sending plain conversation with no tools, so dropping tools cost them nothing. If I had skipped that step I would have spent the night assuming everything was poisoned, or worse, "fixed" five things that were never broken. Scope first, then fix. It is almost never the twenty minutes you regret.

The second thing I found is the one I keep thinking about. That agent had a carefully built fallback chain behind it. If the main model fails, try the second one. If that fails, try a local one. Good design. It had never fired. Not once in three weeks.

It never fired because fallbacks trigger on failure, and nothing had failed. The proxy was returning a perfectly healthy success response with fabricated content inside it. From the agent's point of view the primary was working beautifully every single time. I had built a safety net and then made absolutely certain nothing would ever fall into it.

Fail loud is not a nicety. It is the precondition for every piece of redundancy you own. A component that degrades quietly does not just fail, it disables the machinery you built to survive its failure.

The honest part: I wrote the proxy. Nobody snuck this in. I wrote something that forwarded five fields and did not think about the sixth, and the actual fix was a few lines to pass the tools through and hand back what the upstream said instead of rebuilding the answer myself. Weeks of confidently wrong output from a small omission in code I was proud of.

So here is the lesson, and it is the one this whole series has been circling. Trust is not a status you read. Trust is a test you run. A green light tells you a thing believes it is fine. Only a ground truth test, where you already know the answer and the system cannot possibly guess it, tells you whether it is.

The Build

How to prove your agent can actually do the things it claims, find out who else a shared component broke, and make failure loud enough that your fallbacks can do their job. All of this is generic. It applies to any agent behind any proxy or gateway.

1. Run the needle test

Give the agent a task whose correct answer requires the capability you are testing, and whose answer is a value it could not possibly guess. Generate the value yourself so you already know the truth. If the answer comes back wrong, the capability is not there, no matter what the status page says.

# make a needle only a real fetch can find
NEEDLE="nd-$(head -c 8 /dev/urandom | od -An -tx1 | tr -d ' \n')"
echo "$NEEDLE" > /var/www/html/needle.txt

# now ask the agent to read that URL and report the value
# then compare its answer to $NEEDLE. no match = it never fetched.

Two details make this bulletproof. Use a fresh value every run so a cached or memorised answer cannot pass. And run it once against a target that is deliberately unreachable, because an agent that invents a plausible value for a page that does not exist is telling you everything you need to know.

2. Prove the capability actually reaches the model

The needle test tells you something is wrong. This tells you where. Send a request with a tool definition directly at your gateway and look at what comes back. A working path returns a structured tool call. A broken path returns ordinary prose that merely describes calling the tool.

curl -s "$GATEWAY/v1/chat/completions" \
  -H 'Content-Type: application/json' \
  -d '{"model":"your-model",
       "messages":[{"role":"user","content":"Call the lookup tool. You must use it."}],
       "tools":[{"type":"function","function":{"name":"lookup",
         "parameters":{"type":"object","properties":{"q":{"type":"string"}}}}}],
       "tool_choice":"auto"}' \
| jq '{finish: .choices[0].finish_reason, calls: .choices[0].message.tool_calls}'

# healthy:
{ "finish": "tool_calls", "calls": [ { "function": { "name": "lookup" } } ] }

# broken, and this is exactly what I saw:
{ "finish": "stop", "calls": null }        # with prose describing the call

Then run the same request straight at whatever sits behind your proxy. If it works there and not through the proxy, you have found your culprit and you did it in two commands.

3. Enumerate the blast radius before you change anything

When the broken component is shared, the question is not "how do I fix it" but "who is standing on it". Find every consumer, then sort them by whether they actually use the thing that broke. Most will not.

# who points at this gateway at all?
grep -rIl --exclude-dir=.git "$GATEWAY_HOST" /etc /opt ~/ 2>/dev/null

# of those, who actually sends tools? that is the affected set.
for f in $(grep -rIl "$GATEWAY_HOST" ~/ 2>/dev/null); do
  printf '%s tools=%s\n' "$f" "$(grep -c '"tools"' "$f")"
done

Also check what the component is bound to. A service listening only on the local machine cannot have affected anything on another host, and that single fact can eliminate most of your fleet from the investigation in one command.

ss -ltnp | grep "$PORT"     # 127.0.0.1 means local callers only

4. Make failure loud so your fallbacks can fire

This is the structural fix. A proxy must relay, never invent. Pass the upstream response back as it arrived, and when something goes wrong return a real error rather than a healthy response wrapped around an apology.

# wrong: synthesise a 200 and hand back something plausible
return 200, {"choices": [{"message": {"content": text},
                          "finish_reason": "stop"}]}

# right: relay verbatim, and let real failures be failures
try:
    status, body = upstream(request)     # forward everything, including tools
    return status, body                  # content, tool calls, finish reason, as-is
except Exception as e:
    return 502, {"error": {"message": f"upstream failed: {e}"}}

Check your own middleware for this shape. Anything that catches an error and returns a cheerful default is quietly switching off the redundancy you paid for.

5. Use the needle test as your acceptance check

Do not declare the fix good because the code looks right. Re-run the exact test that caught the problem, and require the real value back. Mine now passes with the token matching exactly, and the request even shows a different client signature from my own, which proves the agent went and fetched it independently rather than echoing anything I had.

6. Put the test on a schedule

The reason this lasted three weeks is that nothing was checking. Run the needle test on a timer and alert when it fails. It costs one request and it is the only thing standing between you and a system that has quietly stopped being able to do its job.

# daily, and shout if the needle does not come back
0 7 * * * /usr/local/bin/needle-test.sh || notify "agent capability check FAILED"

Related: this is part five of a series. Start at the assistant that lies, then the green alert, the lying monitor, and the self-strangling health check. Same disease, five different costumes.