AI Hallucination: What It Means and How to Prevent It

AI hallucination is a plausible but false answer that an AI model presents as fact. For a retailer, that can mean an invented refund rule or a product claim the catalogue doesn’t support. You reduce the risk by giving the agent current store information and requiring it to stop when that information cannot answer the question.

What AI Hallucination Means

An AI hallucination is generated information that sounds convincing but lacks factual support. It can be a made-up source, an invented product detail or a false claim about what a document says. Fluent wording makes the answer easy to accept even when its contents are wrong.

An ordinary error can come from a typo, a calculation mistake or an outdated stock feed. A hallucination involves the model generating unsupported content. Both can mislead a shopper, but fixing the feed alone won’t stop an agent from inventing a policy that the feed never contained.

In a hypothetical store, the catalogue lists a jacket’s fabric but gives no waterproof rating. An answer that assigns the jacket a waterproof rating invents a fact. An answer that says the rating isn’t listed, then shows products with documented ratings, keeps the recommendation within the available information.

AI Hallucination Examples

The Air Canada chatbot case shows how an incorrect policy answer can create a real customer dispute. In Moffatt v Air Canada, 2024 BCCRT 149, the website chatbot said a passenger could claim a bereavement fare after travel. The linked policy page said the opposite.

The British Columbia Civil Resolution Tribunal found Air Canada liable for negligent misrepresentation. Deeth Williams Wall’s account of the ruling explains that the airline remained responsible for information on its website. Providing a link to the correct policy did not excuse the chatbot’s incorrect answer. The decision establishes the consequences of inaccurate chatbot advice, without establishing which model or technical failure produced it.

In 2025, Deloitte’s report for Australia’s Department of Employment and Workplace Relations contained fabricated references and an apparent AI-generated quotation from a federal court judgment. Associated Press reported that Deloitte agreed to repay the final contract instalment. A revised report disclosed generative AI use, and the department said its recommendations remained unchanged.

The government report page identifies the work as an assurance review of the Targeted Compliance Framework. The fabricated references show why a source-looking citation is insufficient: the cited work must exist and support the claim.

Why Language Models Hallucinate

Language models generate text by predicting likely next words from patterns learned during training. That process can produce a convincing sentence without establishing that the sentence is true. Looking up store data gives the model evidence to use, but the generated answer still needs to match that evidence.

Several conditions make unsupported answers more likely:

• The training data contains little information about the subject, especially a rare fact or specialised product.
• The question asks for something outside the model’s knowledge, such as your store’s unpublished return exception.
• A vague or leading prompt invites the model to fill a gap instead of asking a question.
• The retrieved document is incomplete, outdated or about a different product variant.

OpenAI’s 2025 research on hallucinations explains another cause: training and evaluation can reward guesses more than acknowledging uncertainty. A model can earn credit for a lucky guess while giving a wrong answer when the same approach fails. For shopper answers, the system needs to accept an unanswered question as a valid outcome.

How Often AI Hallucinates

There is no single AI hallucination rate that applies to every model or store. Published figures measure a particular task under particular conditions. An incorrect-answer rate, a fabricated-citation rate and the share of factual claims with errors have different denominators.

In a 2023 study published in Scientific Reports, William Walters and Esther Isabelle Wilder tested ChatGPT using GPT-3.5 and GPT-4. They generated literature reviews on 42 topics and checked 636 citations across 84 reviews. The study found that 55% of GPT-3.5 citations and 18% of GPT-4 citations were fabricated. Those figures measure citation generation by those versions, not all answers from ChatGPT or a current shopping agent.

OpenAI’s 2025 SimpleQA results show how abstaining changes the risk on short factual questions.

Model	Correct answers	Incorrect answers	Abstentions
gpt-5-thinking-mini	22%	26%	52%
OpenAI o4-mini	24%	75%	1%

These are model-specific benchmark results, not a general accuracy score for the ChatGPT app. The model that abstained more often gave far fewer wrong answers.

ChatGPT’s accuracy depends on the model, the question and whether it has useful evidence or tools. Writing a readable product summary is a different task from supplying an exact stock count. A published benchmark cannot tell you whether an agent will follow your refund policy.

EY’s 2026 paper, Managing hallucination risk in LLM deployments at the EY organization, addresses risk across the AI pipeline. Its scope includes professional services such as audit and tax, so it isn’t a measured failure rate for retail conversations.

PwC’s AI hallucinations: what business leaders should know, by Ege Gürdeniz and Ilana Golbin Blumenfeld, discusses inadequate data and unsuitable tasks. It recommends access to an authoritative knowledge base and human review. Those recommendations support practical safeguards without supplying a universal hallucination percentage.

The AI Hallucination Problem for Retailers

Invented shopper answers can change a purchase decision and leave your staff handling the consequences. A customer might buy a product because an agent invents a feature, or expect a refund because it states an exception your policy doesn’t contain. A false stock answer can also promise goods you cannot supply.

Under Australian Consumer Law, misleading claims create legal risk even when the business did not intend to mislead. The Australian Competition and Consumer Commission’s guidance on false or misleading claims covers prices and stock availability. It also covers claims about consumer rights and remedies, across websites and other business communications.

Treat chatbot answers as customer-facing store information. A warning that AI can make mistakes does not make a false product statement accurate. Your refund answers must also distinguish your change-of-mind policy from customers’ rights when goods are faulty. Read the guide to AI regulation in Australia for the wider rules affecting AI use.

How to Reduce Hallucinations in Shopper Answers

Ground shopper answers in the current catalogue and store policies, then prevent the agent from filling unsupported gaps. Grounding means supplying the relevant facts at answer time. It reduces opportunities to invent facts, but it does not guarantee an error-free answer.

Use separate controls for the information and the answer:

1. Connect product answers to the correct variant and current price and stock records. Keep policy documents current and remove conflicting versions. The guide to an AI knowledge base explains how to organise the documents an agent needs.
2. Require a supporting product field or policy passage for claims that affect a purchase. A retrieved document must actually contain the answer. For calculations or live availability, use the store system’s result directly.
3. Define what happens when evidence is missing. The agent can ask which variant the shopper means, say a detail isn’t listed or pass the question to staff. It must avoid inventing a return exception to complete the conversation.
4. Test uncertain questions as well as routine ones. Include a product with a missing specification and a customer asking for an unsupported refund promise. Repeat relevant checks after policy, catalogue or model changes.
5. Review incorrect conversations and correct the cause. Fix bad source data, retrieval of the wrong document or instructions that encourage guessing. Keep staff able to take over consequential disputes.

In the hypothetical jacket example, the acceptance check is concrete. Ask for the waterproof rating when the catalogue has none. The agent must avoid supplying a rating and explain the missing detail or find an alternative with documented specifications. Use the chatbot testing guide to turn those store-specific questions into repeatable checks before shoppers rely on the answers.