Before you trust an AI answer, check these four things
A plant scientist's routine for spotting AI hallucinations: which claims to check first, how to check them cheaply, and when to stop.
TOOLS IN AIBASICS OF AI
Dhruv Aditya Srivastava
11/21/20263 min read
I use AI tools most days, for R code, for summarizing papers, for first drafts of methods sections. Most of the time the output is fine. The problem is the times it isn't, because a wrong answer from a language model reads exactly like a right one. It has the same confident tone and the same tidy formatting, and nothing marks the part that's made up.
Scientists already have a habit for this. We don't trust a result until we know where it came from. This post is about applying that habit to AI answers without spending the whole afternoon on it.
Not every sentence needs checking
If you try to verify every line of a long answer, you'll give up by the third paragraph. So don't. Most sentences in an AI answer are framing, opinion, or general background that's either obviously true or doesn't matter much. The ones that can hurt you are the claims that are specific and checkable:
Numbers. Concentrations, p-values, sample sizes, percentages, dates.
Citations. Author, year, journal, DOI.
Named things. Gene names, software packages, functions, protocols, people.
Anything computed. A number the model says it got by doing math or running code.
These are the claims that end up in a figure legend, a grant, or a script, and they're the ones a model is most likely to get subtly wrong.
Citations: assume nothing
Made-up references are the best-documented failure. Walters and Wilder asked ChatGPT to write short literature reviews on 42 topics and then checked all 636 citations it produced. About 55% of the GPT-3.5 citations didn't exist, and about 18% of the GPT-4 ones didn't either. Among the citations that were real, many still had errors in details like the authors, year or pages (Walters & Wilder, 2023, Scientific Reports).
Newer models are better than that, but better isn't the same as safe. My rule is simple. If a citation is going into anything I'll share, I look it up myself. A DOI lookup takes ten seconds. If the model gives a paper with no DOI, I search the title in PubMed or Google Scholar. If I can't find it in a minute, I treat it as fake.
Package and function names: check before you install
This one matters for anyone who uses AI to write code. Models sometimes suggest software packages that don't exist. Spracklen and colleagues tested 16 code-generating models on 576,000 code samples in Python and JavaScript and found that these invented package names come up often. They also tend to repeat (Spracklen et al., USENIX Security 2025). That repetition is the security risk. Someone can register a malicious package under a name the model keeps inventing, and people who copy the install line get the malware.
So before you run pip install or install.packages() on something you've never heard of, look it up on PyPI, CRAN or Bioconductor. Check that it has a real repository, some history, and more than a handful of downloads.
Numbers: recompute, don't reread
If the model says it calculated something, don't reread its explanation and nod. Run the calculation yourself. For anything beyond simple arithmetic, ask the model for the code, run it on your own machine, and compare. A normalization step or a fold-change can look completely reasonable in prose and still be off by a factor of ten.
Checking against your own data works the same way. If the model summarized your CSV, pick two or three of the numbers it quoted and find them in the file.
Long chats drift
The longer a conversation goes, the more care it needs. Liu and colleagues showed that language models use information at the beginning and end of a long input better than information buried in the middle, even in models built for long contexts (Liu et al., 2024, TACL). In practice, the instruction you gave forty messages ago, or the table you pasted at the start, is easier for the model to lose track of.
When a chat gets long and the stakes are real, I either start a fresh session with a short summary of what matters, or I ask the model to restate the key facts it's relying on so I can check them.
Make it a habit, not a project
I got tired of doing these checks by hand, so I wrote a small open-source skill called Hallucination Check. It does the triage described above. It pulls out up to eight of the riskiest checkable claims, skips opinions and hedged statements, checks each claim in the cheapest way available, and gives a low, medium or high score along with the claims that failed. It's built for Claude, but the method works with any tool, and you can do it by hand with a notepad.
The point isn't to distrust AI. It's to treat its output the way you'd treat a draft from a new student who's quick, keen and sometimes wrong. You read it, you check the parts that matter, and you sign your name only to what you've verified.
Sources
Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13, 14045. https://doi.org/10.1038/s41598-023-41032-5
Spracklen, J., et al. (2025). We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs. USENIX Security 2025. https://arxiv.org/abs/2406.10279
Liu, N. F., et al. (2024). Lost in the Middle: How Language Models Use Long Contexts. TACL, 12, 157-173. https://doi.org/10.1162/tacl_a_00638
Insights
Explore AI news, expert articles, and workshops.
Connect
Learn
© 2026 The AI Advances. All rights reserved.
