Completeness, accuracy, relevancy, and efficiency are what we look for in the prompt responses we receive from AI models. While all AI models may strive to produce high-quality and high-value responses from user prompts submitted, AI is still vulnerable to errors, inaccuracies, and misunderstandings, all of which impact its responses and can create “hallucinations”.
Today, we’ll be taking a closer look at hallucinations, why they occur, and how to limit them, especially if you’re in the process of training an AI model.
What are AI hallucinations and why do they happen?
An AI model hallucinates when it generates text that is incorrect, nonsensical, or made up. Some examples of AI hallucinations include a model answering questions about the weather forecast in a fictional city and generating fabricated references in an academic research paper.
Hallucinations can occur for several reasons. Aside from the “garbage in, garbage out” approach to the quality of inputted data, LLMs are also static. They’re essentially “frozen in time” and lack up-to-date information that can only be internalized from another round of facilitated training.
LLMs may lack domain-specific knowledge, meaning they’re trained for generalized tasks and do not know a company’s private data.
Types of hallucinations
Based on the user input contained within a prompt, three primary kinds of hallucinations can occur: input-conflicting hallucinations, context-conflicting hallucinations, and fact-conflicting hallucinations.
- Input-conflicting hallucinations – occur when the content generated from the LLM deviates or is in conflict with the user input.
- Context-conflicting hallucinations – occur when the generated content contradicts itself within the output.
- Fact-conflicting hallucinations – occur when an LLM generates factually inaccurate or untrue content.
Let’s use an example to explore each kind of hallucination. Let’s say you type a prompt into an LLM that says, “Can you recommend a meal for a delicious dinner?”
The LLM produces this response, “Yes, here is a meal that will be a delicious lunch. How about fried chicken with mashed potatoes? In addition, tomatoes are an excellent pairing for this dish, as they are rich in calcium. Enjoy this steak!”
The above response contains all three examples of AI hallucinations. The model recommending a meal for lunch instead of dinner is an input-conflicting hallucination. Tomatoes are not, in fact, rich in calcium, making this a fact-conflicting hallucination.
Finally, the response is based on the suggestion of fried chicken and mashed potatoes as the meal but, at the end, the model suddenly mentions steak with no prior context. This is an example of a context-conflicting hallucination.
How to measure hallucinations
You can measure the accuracy of an LLM’s output by using a measuring method called FActScore (Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation). FActScore is a tool for quantifying the accuracy of LLM outputs by breaking down content into “atomic facts” and assessing each fact against verified sources. The FActScore will then generate an accuracy percentage that weighs the overall accuracy of the model’s response.
Another way to measure hallucinations is the Med-HALT approach (medical domain hallucination test for large language models) which proposes a two-tiered approach to evaluate the presence and impact of hallucinations in biomedical-generated LLM outputs. The two-tiered approach is made up of:
- Reasoning Hallucination Tests
- Memory Hallucination tests
Methods for preventing and limiting AI hallucinations
Retrieval Augmented Generation (RAG) is a well-established method of controlling and minimizing the chance of hallucinations occurring. RAG helps reduce the possibility of hallucinations occurring by augmenting LLMs with updated and unavailable information (including private company data) for content generation. It makes LLMs more dynamic in the process.
It also offloads the factual information to the retrieval system and LLMs only have to generate a response from the contexts provided.
Another prominent method is Chain-of-Verification (CoVe) which introduces a four-step process that enables LLMs to draft, verify, and refine their responses.
The four-step CoVe procedure consists of:
- Generating a baseline response – the LLM is given a query and generates its initial response.
- Planning verification questions – the model uses the query and baseline responses to generate a list of verification questions that helps it to self-analyze its responses to identify errors.
- Execute verifications – the LLM answers each verification question in turn and checks the answer against the original response to check for errors and inconsistencies.
- Generate final verified response – having verified and fact-checked its content, the model generates a revised response incorporating the verification results.
Other ways to limit AI hallucinations include limiting the model’s outcomes to a certain fixed form, i.e. prompting a model to only generate a class label instead of free-form text, creating highly-specified prompts, and, most importantly, always keeping a human in the loop.
Human reviewers add domain expertise to the process, evaluate generated content, and ensure the model’s outputs meet the desired criteria.
Conclusion
As AI becomes more widespread across industries, including life sciences, it’s crucial for organizations building in-house LLMs to build and utilize AI systems responsibly. AI is not and never will be a replacement for humans. It augments human intelligence but still needs to be monitored and guided by human experts.
If you’d like to learn more about Saama’s suite of proprietary AI tools and how they can elevate your workflows, book a demo with us.
