Every business owner who's tried an AI chatbot has the same story: it was impressive for a week, then it confidently told a customer something untrue.
Why it happens
A language model's job is to produce a plausible next sentence. Left to its own devices it will always produce one — whether or not it has any basis for it. Ask a general-purpose model about your refund policy and it will invent something that sounds like a refund policy, because that's the shape of the expected answer.
The model isn't lying. It was never given anything true to say.
The fix: retrieval, then answering
The approach that works is unglamorous. Before answering, search your actual content for the passages relevant to the question, and instruct the model to answer only from those. If nothing relevant comes back, it says so and offers to pass the visitor to a person.
- •Your content is the source of truth, not the model's memory.
- •Each answer is built from passages that actually exist.
- •"I'm not sure — let me put you through to the team" is a correct answer, not a failure.
How to test it in two minutes
Ask any assistant a question your content definitely doesn't cover — a made-up product, a branch you don't have. A grounded assistant admits it. An ungrounded one invents something, and you've learned everything you need to know before it does that to a paying customer.