Show HN: LettuceDetect – Lightweight hallucination detector for RAG pipelines
github.comHallucinations are still a major blocker for deploying reliable retrieval-augmented generation (RAG) systems, especially in complex domains like medical or legal.
Most existing hallucination detectors rely on full LLM inference (expensive, slow), or struggle with long-context inputs.
I built LettuceDetect — an open-source, encoder-only framework that detects hallucinated spans in LLM-generated answers based on the retrieved context. No LLMs needed, and it much more efficiently.
Highlights:
- Token-level hallucination detection (unsupported spans flagged based on retrieved evidence)
- Built on ModernBERT — handles up to 4K token contexts
- 79.22% F1 on the RAGTruth benchmark (beats previous encoder models, competitive with LLMs)
- MIT licensed
— Includes Python packages, pretrained models, and Hugging Face demo
GitHub: https://github.com/KRLabsOrg/LettuceDetect
Blog: https://huggingface.co/blog/adaamko/lettucedetect
Preprint: https://arxiv.org/abs/2502.17125
Models/Demo: https://huggingface.co/KRLabsOrg
Would love feedback from anyone working on RAG, hallucination detection, or efficient LLM evaluation. Also exploring real-time hallucination detection (vs. just post-gen) — open to thoughts/collab there.
if you have the knowledge to detect your own hallucinations, then you have the knowledge to not hallucinate in the first place.
the fact that we keep seeing "hallucination detectors" means the system is hopelessly broken. And products like these are usually snake oil, imo.