NOT IN CORPUS — teaching a local model to abstain instead of hallucinate

I asked my local RAG stack about my own files and it hallucinated. Confidently. The corpus was mine, the retrieval was correct, the small model just… invented an answer around thin context, in fluent English.

我询问了我的本地 RAG 堆栈关于我自己的文件，结果它产生了幻觉。而且非常自信。语料库是我的，检索也是正确的，但这个小模型只是……在稀薄的上下文中编造了一个答案，而且英语表达还很流利。

The fix wasn’t a bigger model. It was a behavioral contract, enforced in the prompt and proven in testing: text You are a grounded assistant. Use ONLY the context below. Rules: (1) If the context does not contain the answer, reply exactly: NOT IN CORPUS. (2) Cite which source title each fact came from. (3) Do not use outside knowledge.

解决办法不是换一个更大的模型。而是一份行为契约，通过提示词强制执行并在测试中得到验证：文本“你是一个基于事实的助手。仅使用下方的上下文。规则：（1）如果上下文中不包含答案，请准确回复：NOT IN CORPUS。（2）引用每个事实来源的标题。（3）不要使用外部知识。”

The result: the same query that hallucinated now replies “NOT IN CORPUS.” — and when the context is there, the answer arrives with source citations attached. Two things I learned building this: The retrieval was never the problem. Vector search found the right neighborhood every time.

结果是：同一个产生幻觉的查询现在回复“NOT IN CORPUS。”——而当上下文存在时，答案会附带来源引用。在构建这个系统的过程中，我学到了两件事：检索从来都不是问题所在。向量搜索每次都能找到正确的范围。

The mouth was the problem — a small model treating thin context as an invitation to improvise. People blame RAG for hallucinations that are actually generation-layer failures. Abstention is a feature you have to test.

问题出在“嘴巴”上——一个小模型把稀薄的上下文当作了即兴发挥的邀请。人们将幻觉归咎于 RAG，但实际上那是生成层面的失败。弃权（拒绝回答）是一个你必须进行测试的功能。

I now run queries I know are absent from the corpus, specifically to check the model refuses. An abstention you haven’t tested is an abstention you don’t have. The stack: Ollama + nomic-embed-text + ChromaDB + granite 3B, fully local on an 8GB MacBook Air.

我现在会运行一些我知道语料库中不存在的查询，专门检查模型是否会拒绝。一个未经测试的弃权功能等于没有。技术栈：Ollama + nomic-embed-text + ChromaDB + granite 3B，完全在 8GB 内存的 MacBook Air 上本地运行。

300K+ chunks of my own archive, no cloud, no API, no accounts. The orchestrator dispatches each question to four workers and the EvidenceFlow guard verifies before anything reaches the screen. My full archive answers questions in my own voice now — and it has permission to say “I don’t know.” That permission is the feature.

超过 30 万个我个人归档的文本块，无需云端、无需 API、无需账户。编排器将每个问题分发给四个工作节点，EvidenceFlow 防护层会在任何内容显示在屏幕上之前进行验证。我的完整归档现在能用我自己的语气回答问题——而且它被允许说“我不知道”。这种许可本身就是一项功能。

The tooling is published: ask-mirror — hybrid retrieval (vector + SQL) with the abstention guard built in, AGPL. The verification discipline comes from EvidenceFlow — fail-closed on Ship 5, now guarding the whole stack.

工具已发布：ask-mirror——内置弃权防护的混合检索（向量 + SQL），采用 AGPL 协议。验证准则来自 EvidenceFlow——在 Ship 5 上实现故障关闭，现在守护着整个堆栈。

Create template Templates let you quickly answer FAQs or store snippets for re-use. Submit Preview Dismiss Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment’s permalink. Hide child comments as well Confirm For further actions, you may consider blocking this person and/or reporting abuse

创建模板 模板让您可以快速回答常见问题或存储片段以供重复使用。提交 预览 取消 您确定要隐藏此评论吗？它将在您的帖子中被隐藏，但仍可通过评论的永久链接查看。同时隐藏子评论 确认 如需进一步操作，您可以考虑屏蔽此人或举报滥用行为。