Trust the Calculator
Trust the Calculator / 相信计算器
The pricing formulas in Motor, the estimating engine I built for a water feature shop, did not come from the manual. I pulled 32 of them out of the JavaScript behind Aquascape’s contractor calculator, the tool contractors actually use to bid jobs. The manual was sitting right there, official and free. Ignoring it was the best design decision in the whole system.
我在为一家水景店构建的估算引擎 Motor 中使用的定价公式,并非来自手册。我从 Aquascape 承包商计算器背后的 JavaScript 代码中提取了 32 个公式,这才是承包商在实际投标时使用的工具。手册就摆在那里,既官方又免费。但忽略它,是我整个系统设计中最明智的决定。
A vendor never ships a sloppy calculator. Why trust the calculator over the manual? Because of what happens when each one is wrong. If the manual sizes a pump wrong, a reader shrugs and moves on. If the calculator sizes a pump wrong, a contractor bids a job at that number, wins it, and loses money on the install. Then the phone rings. So calculators get fixed and manuals drift. Give it ten years and the two quietly disagree, and everyone in the trade knows which one to trust without anyone saying so. A vendor will ship a sloppy PDF. They will never ship a sloppy calculator.
供应商绝不会发布一个草率的计算器。为什么要相信计算器而不是手册?因为当两者出错时,后果截然不同。如果手册上的泵规格算错了,读者只会耸耸肩,翻篇了事。但如果计算器算错了,承包商就会按那个数字投标,中标后,安装工程就会亏钱。接着,投诉电话就会打进来。因此,计算器会被不断修正,而手册则会逐渐过时。十年过去,两者悄然产生分歧,业内人士无需多言,心里都清楚该信哪一个。供应商可能会发布一份草率的 PDF 文档,但他们绝不会发布一个草率的计算器。
Documentation is what a domain says about itself. The artifacts money flows through are what it actually believes. Once you see that split, you cannot stop seeing it. The other half was in old invoices. Formulas only get you to cost. What a shop charges on top of cost is a belief about its market, and no vendor document holds that number. So I pulled 132 historical quotes out of the shop’s CRM. Real quotes, sent to real customers, most of them paid. I calibrated Motor’s markup against those, then checked its output against what the shop had actually charged.
文档是一个领域对自身的描述,而资金流经的产物才是它真正信奉的东西。一旦你看到了这种分裂,你就再也无法忽视它。另一半真相隐藏在旧发票里。公式只能帮你算出成本,而一家店在成本之上收取的费用,代表了它对市场的判断,没有任何供应商文档会记录这个数字。因此,我从该店的 CRM 系统中提取了 132 份历史报价。这些都是发给真实客户的真实报价,且大部分已结清。我根据这些数据校准了 Motor 的加价逻辑,然后将其输出结果与该店实际收取的费用进行了比对。
The result: Calibrated against 132 real quotes, Motor’s estimates landed within 5 percent of what the shop actually charged, with no pricing rule taken from documentation. I could have just asked the owner what his markup was. But what an owner says and what his invoices show are rarely the same number, and the invoices are the ones customers paid. When the two disagree, believe the invoices.
结果显示:在经过 132 份真实报价的校准后,Motor 的估算结果与该店实际收费的误差控制在 5% 以内,且没有使用任何来自文档的定价规则。我本可以直接问店主他的加价比例是多少,但店主嘴里说的和发票上显示的往往不是同一个数字,而发票才是客户真正支付的金额。当两者不一致时,请相信发票。
The same bug in a different industry. I build and run systems in several industries, and the surprising part is how often the same small idea matters in all of them. This year it showed up in medical billing. I run a denial engine that reads insurance denials and recommends the next move: appeal, fix the coding and resubmit, bill the patient, or write it off. The first version reasoned from guidance prose, appeal-strategy text that reads like documentation. On a golden set of 30 adjudicated cases, it picked the right action 36.7 percent of the time.
不同行业中的同一个“漏洞”。我在多个行业构建和运行系统,令人惊讶的是,同一个小小的理念在所有行业中都至关重要。今年,它出现在医疗账单领域。我运行着一个拒付处理引擎,它读取保险拒付信息并推荐下一步操作:申诉、修正编码后重新提交、向患者收费或核销。第一个版本是基于指导性文字进行推理的,那些申诉策略文本读起来就像文档。在一组 30 个已裁决案例的黄金测试集中,它选对操作的概率仅为 36.7%。
What broke: The engine kept recommending appeals for coding errors that should just be fixed and resubmitted, because the guidance said appeal and the model believed it. It scored 36.7 percent on action selection. I had banned documentation from my inputs, then smuggled it back in as a prompt. The fix was the same move as the calculator. We keep a reference layer that records what experienced billers actually do with each denial code, distilled from real worked claims. I anchored the engine to that instead of the prose. Action accuracy went from 36.7 percent to 76.7 percent in one evening, and classification went from 80 percent to 93.3 percent. The model did not get smarter. The spec got real.
问题出在哪:引擎不断建议对本应修正并重新提交的编码错误进行申诉,因为指导文档说要申诉,而模型信了。它在操作选择上的得分只有 36.7%。我本已禁止将文档作为输入,却又将其作为提示词偷偷塞了进去。修复方法与计算器案例如出一辙:我们建立了一个参考层,记录经验丰富的计费员针对每个拒付代码的实际操作,这些数据是从真实处理过的索赔中提炼出来的。我将引擎锚定在这些数据上,而不是那些文字描述。仅仅一个晚上,操作准确率就从 36.7% 提升到了 76.7%,分类准确率从 80% 提升到了 93.3%。模型并没有变聪明,是规格说明变得真实了。
Key Metrics / 关键指标
-
32 formulas pulled from the contractor calculator’s JavaScript
-
132 real quotes used to calibrate markup
-
5% gap between Motor’s estimates and actual shop pricing
-
76.7% denial-action accuracy after anchoring to worked claims, up from 36.7%
-
32 个公式:从承包商计算器的 JavaScript 中提取
-
132 份真实报价:用于校准加价逻辑
-
5% 的误差:Motor 估算值与商店实际定价之间的差距
-
76.7% 的拒付处理准确率:锚定真实索赔记录后,从 36.7% 提升而来
Two records / 两套记录
Every domain keeps two records of itself. One is written for readers: manuals, best-practice guides, onboarding docs. The other is written for money: calculators, invoices, worked claims. The first record is what the domain wants to be true. The second is what it paid to learn. When you encode a domain into software, you are choosing which record to believe, whether you know it or not.
每个领域都保留着两套关于自身的记录。一套是写给读者看的:手册、最佳实践指南、入职文档。另一套是为金钱服务的:计算器、发票、处理过的索赔记录。第一套记录是该领域“希望”成为事实的内容,第二套则是它“付出代价”才学到的经验。当你将一个领域编码进软件时,无论你是否意识到,你都在选择相信哪一套记录。
Key insight: Encode the rules a domain risked money on (the shipped calculator, the paid invoice, the worked claim) and treat everything it merely wrote down as a rumor. Pull your rules from the artifacts people bid with. The calculator beats the manual it shipped next to. Calibrate against money that actually moved, and measure the gap in percent, not in vibes. Treat documentation as a hypothesis about the domain, never as the spec. When your model gets a domain wrong, check what you fed it before you blame the model.
核心洞察:将领域内那些投入了真金白银的规则(已发布的计算器、已支付的发票、已处理的索赔)编码进去,而将所有仅仅写在纸面上的东西视为传闻。从人们实际用于投标的产物中提取规则。计算器永远胜过它旁边附带的手册。根据实际流动的资金进行校准,并用百分比来衡量差距,而不是凭感觉。将文档视为关于该领域的假设,永远不要将其视为规格说明。当你的模型在某个领域出错时,在责怪模型之前,先检查你喂给它的数据。
None of this is really about ponds or insurance claims. It is about where truth lives in a domain. A domain will tell you anything in its documentation. What it bids with is what it believes. Build from that. And if you are pulling formulas out of someone’s calculator at midnight, in an industry nothing like mine, we should compare notes.
这一切其实与水池或保险索赔无关,而是关于真相在一个领域中究竟存在于何处。一个领域会在文档中告诉你任何事,但它用什么来投标,它就信什么。以此为基础进行构建吧。如果你也在深夜从某人的计算器里提取公式,且身处一个与我完全不同的行业,我们应该交流一下心得。