How to stop an AI chatbot making things up

Ask a general-purpose AI model a question it cannot answer and it will usually answer anyway. It will sound confident, it will be fluent, and it may be completely wrong. In the industry this is called a hallucination, and it is the single biggest reason business owners hesitate before putting a chatbot in front of customers. Nobody wants a bot inventing a price, a policy or an opening time.

The good news is that hallucinations can be cut dramatically. They are largely a design problem. This article explains why they happen and the six things we build into every chatbot to keep yours inside what you have approved.

Why chatbots make things up

Large language models do not look answers up. They generate the most plausible next words based on patterns learned during training. That is what makes them so natural to talk to, and it is also the flaw: if a question falls outside what the model actually knows, the most plausible-sounding answer is still produced. Plausible and true are not the same thing.

Three situations make this worse. First, when the chatbot has no access to your own information, so it guesses from general knowledge. Second, when it is allowed to answer anything at all, including topics it should hand to a person. Third, when nobody is checking what it said after launch, so small errors become habits.

1. Ground every answer in your approved content

The most effective fix is also the most obvious: give the chatbot your actual information and instruct it to answer only from that. Your services, prices, policies, service areas, opening hours and the FAQs your team answers every week become the chatbot’s knowledge base. When a visitor asks about cancellation terms, the answer comes from your cancellation terms, not from an average of every cancellation policy on the internet.

This is usually called retrieval-augmented generation, or RAG. In practice it means the chatbot finds the relevant passage first, then writes its reply from it. The reply can still be warm and conversational; it is simply anchored to a source.

2. Narrow the scope on purpose

A chatbot that will discuss anything is a chatbot that will eventually be wrong about something. We design each chatbot for a specific job: answering questions about your business and moving a genuine enquiry towards the next step. Questions outside that scope get a polite, honest redirect rather than an improvised answer.

Counter-intuitively, a narrower chatbot feels more helpful, not less. Visitors are not looking for a general assistant on a dental practice website. They want to know whether you can see them this week.

3. Make “I don’t know” a first-class answer

The chatbot needs explicit permission, and explicit wording, to say it does not know. Ours are designed to say something like “That’s one for the team, I’ll make sure someone comes back to you today” and to capture the question and contact details so the hand-over actually happens. An honest gap followed by a proper follow-up builds more trust than a confident guess.

4. Have a person design and approve the conversation

Every chatbot we launch is designed by our team, who have spoken to the business owner, read the website and understood how the team talks to customers. The greetings, the follow-up questions, the tone and the hand-over points are all drafted, reviewed and signed off before anything goes live. The AI handles the variety of live conversations; the script and the boundaries are human.

5. Test against your enquiries before launch

Before switching a chatbot on, we run it against a set of questions pulled from the business’s inbox and call notes, including the awkward ones. Where it answers well, fine. Where it stretches, we tighten the content or the instructions. This step catches most problems before a customer ever sees them.

6. Keep reviewing after launch

Accuracy is not a one-off setting. Prices change, services get added, a new question starts coming up. Part of running a chatbot properly is quality checking transcripts regularly, spotting anything that was answered poorly and fixing the underlying content. The monthly report we send clients includes exactly this: what people asked, what the chatbot said and anything that needs updating.

What this looks like for a visitor

Someone lands on a solar company’s website at 9pm and asks whether a battery can be added to a six-year-old system. The chatbot answers from the company’s own guidance on retrofits, asks the size of the existing system, offers to have an installer call with options and takes a phone number. It does not quote a price it has not been given. It does not promise a rebate it is unsure about. It says what it knows and routes the rest.

That is the standard to hold any chatbot to: warm, useful and honest. If a vendor cannot explain how their chatbot stays inside your approved information, keep asking until they can.