Hallucination rate is the wrong question

What’s your hallucination rate?

I get this question constantly. And for a while, I tried to answer it with benchmarks, percentages, confidence intervals.

None of it moved the needle.

Turns out the question isn’t really “how often does your agent lie?” It’s “should I trust this thing?”

And that’s not something you answer with a number.

It’s something Fred answers.

Every org has a Fred. The senior engineer who’s picky, skeptical, hard to impress. The one whose thumbs-up changes the room’s energy.

So we stopped trying to prove hallucination rates. We added a simple dropdown: rate every response, 1-5.

Now when someone asks “how accurate is the agent?” — I don’t give them a number. I tell them Fred’s giving it a 3.8/5.

THAT lands different.