Asynchronous Form Schema Discovery: Idempotent Retries and Secure Temporary Files

异步表单模式发现：幂等重试与安全临时文件

Use a durable job record as the unit of work for form schema discovery: validate the document at admission, submit one asynchronous extraction job under a correlation ID, poll it with bounded retries, and treat the temporary files on disk as scratch space that nothing downstream is allowed to trust. 将持久化作业记录作为表单模式发现的工作单元：在准入时验证文档，在关联 ID 下提交一个异步提取作业，通过有限重试进行轮询，并将磁盘上的临时文件视为下游绝不可信任的暂存空间。

In a healthtech service that watermarks discharge packets before they go to an outside clinic, the constraint that decides the design shows up before the fidelity-versus-render-cost trade ever does. It is whether you can say, six months later, exactly which bytes left the network and under which policy. Everything below follows from that one requirement. 在一家为出院数据包发送至外部诊所前添加水印的医疗科技服务中，决定设计的约束条件远比“保真度与渲染成本”的权衡更重要。这个约束是：六个月后，你是否能准确说出哪些字节离开了网络，以及是在何种策略下离开的。下文所述的一切都源于这一核心需求。

The failure that pages you: a duplicate packet leaves the building. Two shapes of page dominate cron and queue infrastructure: a job that never ran, and a delivery that ran twice. Field extraction feeding a watermarking step can produce both in the same night, because it sits exactly where request handlers, queues and a slow external job meet. 让你收到报警的故障是：重复的数据包离开了系统。在定时任务（cron）和队列基础设施中，最常见的两种报警形式是：作业从未运行，以及交付运行了两次。为水印步骤提供数据的字段提取过程可能在同一晚产生这两种错误，因为它恰好位于请求处理程序、队列和缓慢的外部作业的交汇处。

Walk the second one through. A worker leases a message for an admitted 14-page referral packet, submits the extraction, and dies before it writes the returned job identifier anywhere durable. The lease expires. A replacement worker reads the same message, finds no job identifier on the correlation record, and submits again — so there are now two extractions, two renders, and, if the delivery step is equally naive, two watermarked packets addressed to the same clinic intake mailbox. Nothing alerts, because every individual call succeeded. 让我们推演一下第二种情况。一个工作进程（worker）租用了一条关于已准入的 14 页转诊数据包的消息，提交了提取任务，但在将返回的作业标识符写入任何持久化存储之前就崩溃了。租约过期，替换的工作进程读取了同一条消息，在关联记录中找不到作业标识符，于是再次提交——现在有了两次提取、两次渲染；如果交付步骤同样简单粗暴，就会向同一个诊所接收邮箱发送两个带水印的数据包。由于每次单独的调用都成功了，系统不会发出任何警报。

The mistake was never a failed request; it was letting a retry mean “start over” instead of “continue”. Standard queues are at-least-once. That is not a defect to work around, it is the contract, and consumer idempotency is the price of admission. Two habits cover most of it: write the job identifier to the correlation record before the first poll, and put a client-supplied idempotency key on the submit so a replayed request resolves to the same job rather than creating a second one. HTTP 429 gets the same treatment — a retried POST that carries an idempotency key is safe to retry; one that doesn’t isn’t. 错误不在于请求失败，而在于将“重试”理解为“重新开始”而非“继续”。标准队列保证“至少一次投递”（at-least-once）。这不是需要规避的缺陷，而是契约，而消费者的幂等性就是准入的代价。两个习惯可以解决大部分问题：在首次轮询前将作业标识符写入关联记录，并在提交时带上客户端提供的幂等键，这样重放的请求会解析为同一个作业，而不是创建第二个。HTTP 429 也是如此处理——携带幂等键的重试 POST 请求是安全的；不带的则不是。

What the manifest must hold before patient data goes out: Validation belongs at admission, before a document occupies a worker slot: MIME type, page count, byte size, checked against policy, with a rejection recorded against the correlation ID and a stable reason returned to the caller. Oversized or malformed uploads that reach the worker pool become a latency amplifier under load, and the amplification is worst exactly when traffic is highest. 在患者数据发出前，清单（manifest）必须包含什么：验证应在准入时进行，即在文档占用工作进程槽位之前：检查 MIME 类型、页数、字节大小是否符合策略，并将拒绝记录在关联 ID 下，同时向调用者返回明确的原因。超大或格式错误的上传如果进入工作进程池，会在负载高时放大延迟，而这种放大在流量最高时最为严重。

Then there is the temporary-file question, which in health data handling is really a retention question. Give each correlation ID its own private working directory created with mode 0700 — Node’s fs.promises.mkdtemp is fine for this — never join a client-supplied filename into a path, and keep inputs and outputs in separate prefixes so an “output” bucket policy can be audited on its own. Delete the scratch directory when the job reaches any terminal state, success or failure. 接下来是临时文件问题，在处理医疗数据时，这实际上是一个保留策略问题。为每个关联 ID 提供其私有的工作目录，权限设为 0700（Node 的 fs.promises.mkdtemp 很适合此操作），永远不要将客户端提供的文件名拼接到路径中，并将输入和输出保存在不同的前缀下，以便可以单独审计“输出”存储桶策略。当作业达到任何终止状态（成功或失败）时，删除暂存目录。

What survives is the manifest. The manifest outlives the files. That is the point. Mine carries the correlation ID, the upstream job identifier, the validated MIME type and page count and size, the input digest, the policy version, the output digest, the render decision, and the timestamps for each transition. The render decision matters more than it looks: this is where the fidelity-versus-cost choice gets recorded. A cheap overlay pass and a full re-render place watermarks differently on a form with tight field geometry, and when someone eventually reports that a signature block was covered, you want to answer from a manifest rather than from a screenshot argument in a chat thread. 最终留存下来的是清单。清单的生命周期比文件更长，这正是重点所在。我的清单包含关联 ID、上游作业标识符、已验证的 MIME 类型、页数和大小、输入摘要、策略版本、输出摘要、渲染决策以及每次转换的时间戳。渲染决策比看起来更重要：这是记录“保真度与成本”权衡的地方。廉价的覆盖层处理和完整的重新渲染在字段布局紧凑的表单上放置水印的方式不同，当有人最终报告签名块被遮挡时，你希望通过清单来回答，而不是在聊天记录中通过截图争论。

Store the manifest where retention is 30 days or longer, per your own policy, and store it apart from the artifacts it describes. 根据你的策略，将清单存储在保留期为 30 天或更长的地方，并将其与它所描述的制品分开存储。

How do I implement form schema discovery with asynchronous jobs, retries, and validation? Submit the extraction as an explicit job with POST /v1/pdf/form/extract, persist the returned identifier, then poll GET /v1/pdf/job/get/{job_id} with exponential backoff, jitter, and a hard deadline taken from your sharing SLO rather than from a number someone typed once. Honour Retry-After when it arrives. No tight loops, and no worker sleeping while holding a lease it could have released. 我该如何实现带有异步作业、重试和验证的表单模式发现？通过 POST /v1/pdf/form/extract 显式提交提取作业，持久化返回的标识符，然后使用指数退避、抖动（jitter）以及基于共享 SLO（而非随意设定的数字）的硬截止时间来轮询 GET /v1/pdf/job/get/{job_id}。当收到 Retry-After 时务必遵守。不要使用紧密循环，也不要让工作进程在持有本可释放的租约时进入睡眠。