When RAG Users Ask Vague Questions: Clarify Once, Learn the Default
When RAG Users Ask Vague Questions: Clarify Once, Learn the Default
当 RAG 用户提出模糊问题时:询问一次,习得默认值
Large Language Models When RAG Users Ask Vague Questions: Clarify Once, Learn the Default Enterprise Document Intelligence [Vol.1 #6bis] – Ask one focused clarification, learn the default from the answer, stay silent next time.
大型语言模型:当 RAG 用户提出模糊问题时:询问一次,习得默认值。企业文档智能 [第 1 卷 #6bis] —— 提出一个有针对性的澄清问题,从回答中习得默认值,下次保持沉默。
This article is a companion in Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks: parsing, question parsing, retrieval, and generation. It extends Article 6 (question parsing) on the case where the question is not precise enough: ask one focused clarification, learn the default from the answer, stay silent next time.
本文是《企业文档智能》系列文章的补充篇。该系列旨在通过四个模块构建企业级 RAG 系统:解析、问题解析、检索和生成。本文是对第 6 篇(问题解析)的扩展,专门探讨当问题不够精确时的处理方案:提出一个有针对性的澄清问题,从回答中习得默认值,下次保持沉默。
The question-parsing brick turns the user’s text into a typed ParsedQuestion. This companion picks up the failure mode that brick names in one bullet and develops it as its own pattern. The question is missing a piece of information the system needs (which document? which page? which clause type?). The cheap fix is to ask. The right fix is to ask, then learn the default so the next case is silent.
问题解析模块将用户的文本转换为类型化的 ParsedQuestion。本文拾起了该模块中仅以要点形式提及的故障模式,并将其发展为一种独立的模式。当问题缺少系统所需的关键信息(哪个文档?哪一页?哪种条款类型?)时,简单的解决方法是询问。而正确的解决方法是:先询问,然后习得默认值,以便下次遇到同样情况时系统能自动处理。
The question-parsing brick sketches the pipeline: the user types free text, question parsing produces a typed ParsedQuestion, the dispatcher routes on the typed fields, retrieval scopes the corpus. The bullet inside that sketch that this companion expands: when ParsedQuestion has a missing or low-confidence field, the system can either (a) silently infer a default, (b) refuse and ask the user, (c) do both with a learned policy. The third option is the production pattern.
问题解析模块勾勒了整个流水线:用户输入自由文本,问题解析生成类型化的 ParsedQuestion,调度器根据类型化字段进行路由,检索模块确定语料库范围。本文重点扩展了其中的一个要点:当 ParsedQuestion 存在缺失或置信度较低的字段时,系统可以(a)静默推断默认值,(b)拒绝并询问用户,或(c)结合习得的策略同时执行两者。第三种选择是生产环境中的最佳实践。
1. The failure mode the main article only mentions
1. 主文章仅提及的故障模式
The question parsing brick covers the happy path. The user types “what is the deductible on Acme Premier?”, the question parser identifies the entity (Acme Premier), the intent (deductible lookup), the schema field to fill (deductible_amount), and the dispatcher routes. Most production traffic does not fit the happy path.
问题解析模块涵盖了“理想路径”。用户输入“Acme Premier 的免赔额是多少?”,问题解析器识别出实体(Acme Premier)、意图(免赔额查询)、需要填充的模式字段(deductible_amount),随后调度器进行路由。然而,大多数生产环境的流量并不符合这种理想路径。
The common failures, on a single uploaded contract, by frequency on real broker traffic:
- Ambiguous field type: “what is the limit?”. Most contracts have several: coverage limit per occurrence, aggregate limit, sub-limit per peril, claim deductible. The system has to guess which one.
- Missing page scope: “what does it say?” on a 200-page document. Where in the document, the summary? the exclusions? the schedule?
- Ambiguous date scope: “what is the deductible on the home contents cover?” on a contract with an old schedule and a renewal endorsement. Which schedule applies?
- Ambiguous intent: “the warranty section”. Read it to cite a clause, summarise it, or extract the conditions?
- Implicit entity: “the policyholder” on a contract that lists a corporate insured, a beneficiary, and an additional named insured.
在处理单个上传合同时,根据真实经纪业务流量的频率,常见的故障包括:
- 字段类型歧义: “限额是多少?”大多数合同包含多个限额:每次事故的承保限额、累计限额、分险种限额、索赔免赔额。系统必须猜测用户指的是哪一个。
- 页面范围缺失: 在 200 页的文档中问“它说了什么?”。文档的哪个部分?摘要?除外责任?还是附表?
- 日期范围歧义: 在一份包含旧附表和续保批单的合同中问“家庭财产保险的免赔额是多少?”。应该适用哪个附表?
- 意图歧义: “保修条款部分”。是需要引用条款、总结内容,还是提取条件?
- 隐式实体: 在一份列出了企业被保险人、受益人和额外指定被保险人的合同中问“投保人是谁?”。用户指的是哪一个角色?
2. The two-Pydantic-schema contract
2. 两个 Pydantic 模式契约
Two structured objects do the work. The first is a ClarificationRequest the system emits when a field on ParsedQuestion is below the confidence threshold. The second is a ClarificationDefault the system stores after each request, so the next equivalent question is answered without asking.
两个结构化对象完成了这项工作。第一个是 ClarificationRequest,当 ParsedQuestion 中的字段置信度低于阈值时,系统会发出该请求。第二个是 ClarificationDefault,系统在每次请求后将其存储,以便下次遇到相同问题时无需询问即可回答。
class ClarificationRequest(BaseModel):
"""Emitted when a ParsedQuestion field is below confidence threshold."""
target_field: str
question_to_user: str
candidate_values: list[str]
proposed_default: str | None = None
proposed_default_reason: str | None = None
class ClarificationDefault(BaseModel):
"""The learned answer, refreshed across requests."""
target_field: str
doctype: str
sub_conditions: dict = Field(default_factory=dict)
candidate_votes: dict[str, float]
confidence: float
sample_size: int
3. The worked broker example
3. 经纪业务案例演示
The clarification loop fires once per request, not once per conversation turn. Each request below is a separate event over time.
澄清循环是针对每个请求触发一次,而不是针对每次对话轮次。以下每个请求都是随时间发生的独立事件。
Case 1: The user uploads a contract and types “qui est l’assureur?” (who is the insurer?). The system has no learned default. It opens a ClarificationRequest: “I will look on page 1, since that is where the insurer is usually named. Is that the right starting point?”. The user clicks Yes. A ClarificationDefault is written: for target_field = insurer_name, source_page = 1 gets a +1 vote.
案例 1: 用户上传合同并输入“qui est l’assureur?”(保险人是谁?)。系统尚无习得的默认值,因此发出 ClarificationRequest:“我将在第 1 页查找,因为保险人通常在该页列出。这是正确的起点吗?”。用户点击“是”。系统随之写入一条 ClarificationDefault:对于 target_field = insurer_name,source_page = 1 获得 +1 票。