What I learned building a compliance answer engine that refuses to make things up

我在构建一个拒绝胡编乱造的合规问答引擎中学到的经验

Swiss business compliance is a genuinely hard information problem. There is no national minimum wage, VAT rules live at the federal level, social-insurance (AHV) admin runs through 26 cantonal offices, and most of the official guidance is only in German or French. If you are a foreign founder setting up a GmbH, you are stitching answers together from a dozen government PDFs. I have been building Canton Compliance Hub, a free multilingual resource that tries to make that legible. The interesting engineering problem was not the content, it was trust: in a domain where a wrong number can cost someone real money, the system has to be citable and it must not hallucinate. Here is what actually worked.

瑞士的商业合规是一个真正困难的信息难题。这里没有全国统一的最低工资标准，增值税规则由联邦层面制定，社会保险（AHV）的管理则分散在 26 个州级办公室，且大多数官方指南仅提供德语或法语版本。如果你是一位正在设立 GmbH（有限责任公司）的外国创始人，你必须从十几份政府 PDF 文件中拼凑答案。我一直在构建“Canton Compliance Hub”，这是一个旨在让这些信息变得清晰易懂的免费多语言资源库。这里有趣的工程问题不在于内容，而在于信任：在一个错误的数字可能导致真金白银损失的领域，系统必须具备可引用性，且绝对不能产生幻觉。以下是我的实战经验。

Two-layer retrieval beats one

双层检索优于单层检索

The naive RAG setup is: chunk documents, embed them, retrieve top-k, stuff into the prompt. It works until a legal term like AHV or MWST needs an exact match that a dense vector glosses over. I run two layers on Postgres with pgvector: Raw chunks are the ground truth. They keep the original wording and the source URL, so every answer can point back to a specific government page. Fact cards are distilled atoms extracted from those chunks, embedded separately, and queried first. Retrieval is hybrid: a BM25 pass catches the exact legal acronyms, a vector pass catches the paraphrases, and a metadata filter (canton, topic, language) runs before ranking so a Geneva question never pulls a Zurich-only rule.

原始的 RAG（检索增强生成）设置是：将文档分块、向量化、检索 Top-k，然后塞入提示词中。这种方法在遇到像 AHV 或 MWST 这样需要精确匹配的法律术语时就会失效，因为稠密向量往往会忽略这些细节。我在 Postgres 上使用 pgvector 运行了两层检索：原始分块作为“事实真理”。它们保留了原始措辞和来源 URL，因此每个答案都能追溯到具体的政府页面。事实卡片（Fact cards）是从这些分块中提取的精炼原子信息，它们被单独向量化并优先查询。检索采用混合模式：BM25 检索用于捕捉精确的法律缩写，向量检索用于捕捉释义，元数据过滤器（州、主题、语言）在排序前运行，确保关于日内瓦的问题永远不会调取仅适用于苏黎世的规则。

Make hallucination structurally hard

从结构上杜绝幻觉

“Please don’t hallucinate” in a system prompt is not a control. What worked was making every factual claim carry a citation marker back to a chunk, then treating any claim without one as a defect. Generation emits inline citation markers. A claim with no marker gets flagged. A cite whose id is not in the retrieved set is a hallucinated citation, and the draft is rejected and retried at temperature 0. For the numbers (rates, thresholds, CHF amounts), a separate verification pass marks each numeric claim supported, unsupported, or contradicting against the retrieved evidence only. Unsupported numbers block publication. That last pass was the highest-leverage thing we added. Cheap draft models are confident and wrong exactly where it matters most in compliance: a plausible-looking rate.

在系统提示词中写“请不要产生幻觉”是无效的控制手段。真正有效的方法是让每一个事实陈述都带有指向原始分块的引用标记，并将任何没有标记的陈述视为缺陷。生成过程会输出内联引用标记。没有标记的陈述会被标记出来。如果引用的 ID 不在检索集合中，则视为幻觉引用，草稿会被拒绝并以 temperature 0 的参数重试。对于数字（费率、阈值、瑞士法郎金额），我们增加了一个单独的验证步骤，仅根据检索到的证据将每个数字陈述标记为“支持”、“不支持”或“矛盾”。不支持的数字会阻止发布。最后这一步是我们增加的最具杠杆作用的环节。廉价的草稿模型往往在合规性最关键的地方表现得自信且错误：给出一个看起来很合理的费率。

Change detection is code, not AI

变更检测应依靠代码，而非 AI

The corpus has to stay current, but re-summarising an unchanged page with an LLM is waste. Change detection is four boring signals: Last-Modified, ETag, an on-page timestamp, and a SHA-256 of the extracted text. The LLM only runs when the content genuinely changed.

语料库必须保持最新，但用大模型去重新总结一个未变更的页面是浪费资源。变更检测依靠四个枯燥的信号：Last-Modified（最后修改时间）、ETag、页面内时间戳以及提取文本的 SHA-256 哈希值。只有当内容确实发生变化时，才会运行大模型。

Ship the primitives as tools

将基础组件作为工具发布

Two byproducts became reusable public artifacts, grounded in official sources and published as open datasets (CC BY 4.0): a minimum-wage-by-canton comparison, and an SME compliance-deadline calendar. The whole retrieval stack sits behind a free compliance self-check: describe your situation in plain language and get a personalised, source-cited overview.

两个副产品成为了可复用的公共资源，它们基于官方来源并以开放数据集（CC BY 4.0）形式发布：一份各州最低工资对比表，以及一份中小企业合规截止日期日历。整个检索栈支撑着一个免费的合规自查工具：用通俗语言描述你的情况，即可获得一份个性化且带有来源引用的概览。

Takeaways

总结

• Two-layer retrieval (atoms first, raw chunks as citation fallback) is worth the extra table.

• Treat a missing or unknown citation as a build error, not a style preference.

• Verify numbers as a separate pass. The draft model will keep inventing them; a checker against your own corpus is the fix.

• 双层检索（先查原子信息，再以原始分块作为引用兜底）非常值得多建一张表。

• 将缺失或未知的引用视为构建错误，而不是风格偏好。

• 将数字验证作为单独的步骤。草稿模型会不断编造数字；针对自身语料库进行校验是解决之道。

If you work with Swiss compliance content and want to compare notes, the site is cantoncompliancehub.ch. Happy to answer questions in the comments.

如果你从事瑞士合规相关工作并希望交流心得，网站地址是 cantoncompliancehub.ch。欢迎在评论区提问。