本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Batch Process Multiple Documents with a Summarization API — Tenant Cost Controls

Short answer: use a tenant-aware batch to process multiple documents for summarization, preserve one accounting record per document, and retry only operations with a clear idempotency boundary. Support-ticket summarization is a small data pipeline, not a loop around a summarization endpoint. For a developer-tools team, the useful unit is a tenant-aware batch: collect a bounded set of documents, normalize them, submit work with an idempotency key, record usage before the result is displayed, and make partial failure visible.

简短的回答是：使用具备租户感知能力的批处理来处理多个文档的摘要任务，为每个文档保留一份会计记录，并且仅对具有明确幂等性边界的操作进行重试。支持工单摘要是一个小型数据流水线，而不是围绕摘要端点的简单循环。对于开发者工具团队而言，有用的单位是具备租户感知能力的批处理：收集一组有限的文档，对其进行标准化，使用幂等键提交任务，在显示结果前记录使用量，并使部分失败的情况可见。

The summary is only one output. The other is an accounting trail that explains which tenant consumed which amount of work. That ordering matters. If I let a retry create a second charge-shaped event, or if I attach usage to the worker instead of the tenant, triage gets faster while the bill becomes impossible to explain. I would rather lose one optional summary than lose the ability to answer, “Which customer caused this usage, and can we reproduce the count?” The ledger is the boundary. Keep it boring. It also forces a useful compromise: source text can expire while hashes and usage records remain available for reconciliation.

摘要只是输出的一部分。另一部分是会计审计追踪，用于说明哪个租户消耗了多少工作量。这种顺序至关重要。如果我允许重试操作创建第二个计费事件，或者将使用量归因于工作节点而非租户，那么虽然分类处理变快了，但账单将变得无法解释。我宁愿丢失一个可选的摘要，也不愿失去回答“是哪个客户导致了此项使用，我们能否复现该统计数据？”的能力。账本就是边界，保持其简单枯燥。这也带来了一个有用的折中方案：源文本可以过期，而哈希值和使用记录则保留以供对账。

What is the bill actually made of? The dominant term in a multi-document job is usually the amount of text sent for processing, not the number of Python function calls. The exact accounting unit depends on the runtime contract, so treat usage as a measured value returned by the processing system or a deliberately defined estimate. Do not infer it from document count. Five short tickets and five exported logs are both five documents, but they are not the same workload. For each document, keep the tenant identifier, document identifier, content hash, input measurement, output measurement when available, processing status, and request id. A batch record stores requested, accepted, completed, and failed counts plus aggregate usage. The raw text belongs in controlled storage; the cost ledger does not need to duplicate it. Here is a deliberately boring manifest. Boring records are easier to audit six weeks later.

账单到底由什么构成？在多文档任务中，主要成本通常是发送处理的文本量，而非 Python 函数调用的次数。确切的会计单位取决于运行时合同，因此应将使用量视为处理系统返回的测量值或人为定义的估算值。不要通过文档数量来推断。五个简短的工单和五个导出的日志虽然都是五个文档，但它们的工作负载并不相同。对于每个文档，应保留租户标识符、文档标识符、内容哈希、输入测量值、可用时的输出测量值、处理状态和请求 ID。批处理记录存储请求数、接受数、完成数、失败数以及汇总的使用量。原始文本应存放在受控存储中；成本账本无需重复存储这些内容。以下是一个刻意保持简单的清单。简单的记录在六周后更容易审计。

from dataclasses import dataclass
from typing import Iterable

@dataclass(frozen=True)
class Document:
    tenant_id: str
    document_id: str
    text: str
    content_hash: str

def build_manifest(documents: Iterable[Document], batch_id: str) -> dict:
    items = []
    for document in documents:
        if not document.text.strip():
            continue
        items.append({
            "batch_id": batch_id,
            "tenant_id": document.tenant_id,
            "document_id": document.document_id,
            "content_hash": document.content_hash,
            "status": "pending",
        })
    return {"batch_id": batch_id, "items": items}

I keep the manifest and usage event longer than the generated prose, subject to privacy and contractual requirements. That lets a later investigation reconstruct what was attempted without keeping every customer sentence forever. Deleting source text makes content-level debugging harder, so the retention policy needs an owner and a reason. How can a batch process multiple documents through a summarization API? A batch worker should not treat the input array as a transaction. One malformed export, timeout, or rate-limit response should move one item to a retryable or terminal state while the rest continues. The batch summary should say so plainly. I model item states as pending, running, succeeded, retryable, and failed. Each transition includes an attempt number and timestamp. A retry is allowed only when the previous attempt is known to have ended without a committed result, or when an idempotency key protects the boundary. Otherwise, a network ambiguity can produce duplicate summaries and duplicate ledger events. Three attempts is a policy choice in the example, not a guarantee from the runtime.

在隐私和合同要求的约束下，我保留清单和使用事件的时间比生成的文本要长。这使得后续调查能够重构之前的尝试过程，而无需永久保存每一句客户的话。删除源文本会增加内容级调试的难度，因此保留策略需要有负责人和明确的理由。批处理如何通过摘要 API 处理多个文档？批处理工作节点不应将输入数组视为一个事务。一个格式错误的导出、超时或速率限制响应应仅将该项移动到可重试或终止状态，而其余部分应继续运行。批处理摘要应明确说明这一点。我将项目状态建模为：待处理、运行中、成功、可重试和失败。每次状态转换都包含尝试次数和时间戳。只有在已知上一次尝试未产生提交结果，或者有幂等键保护边界时，才允许重试。否则，网络的不确定性可能会导致重复的摘要和重复的账本事件。示例中的三次尝试是一种策略选择，而非运行时的保证。

import time
from collections.abc import Callable

def process_batch(items: list[dict], summarize: Callable[[str], dict], max_attempts: int = 3) -> list[dict]:
    results = []
    for item in items:
        for attempt in range(1, max_attempts + 1):
            item["attempt"] = attempt
            item["status"] = "running"
            try:
                response = summarize(item["text"])
                item["status"] = "succeeded"
                item["usage"] = response.get("usage")
                item["summary"] = response["summary"]
                break
            except TimeoutError:
                item["status"] = "retryable"
                if attempt < max_attempts:
                    time.sleep(2 ** (attempt - 1))
            except ValueError as error:
                item["status"] = "failed"
                item["error"] = str(error)
                break
        results.append(item)
    return results

This example leaves concurrency out on purpose. Add a worker pool only after measuring service limits and tenant fairness. A global concurrency setting can let one large tenant occupy every slot while smaller tenants wait. Per-tenant quotas, a shared queue, and a maximum batch size make behavior explainable. Add jitter to production backoff so workers do not retry on the same schedule. The limitation is deliberate: this worker is unsuitable as a complete queue, scheduler, or cost allocator without durable state around it. The first version I would ship has one summary per document. Combining documents can reduce request overhead, but it weakens attribution: a bad result becomes harder to map to a source ticket, and one oversized document can prevent the whole group from completing. Grouping is an optimization, not the default correctness boundary.

此示例特意省略了并发处理。只有在衡量了服务限制和租户公平性之后，才应添加工作池。全局并发设置可能会让一个大型租户占用所有槽位，而较小的租户则在等待。按租户配额、共享队列和最大批处理大小可以使行为变得可解释。在生产环境的退避策略中加入抖动，以防止工作节点在同一时间重试。这种限制是刻意的：如果没有持久化状态的支持，该工作节点不适合作为完整的队列、调度器或成本分配器。我发布的首个版本将为每个文档生成一个摘要。合并文档可以减少请求开销，但会削弱归因能力：错误的结果将更难映射到源工单，且一个超大的文档可能会导致整个组无法完成。分组是一种优化手段，而非默认的正确性边界。

Which question should the operational record answer? The useful question is not only “did the API return 200?” It is “can support explain this result and this usage to one tenant?” A successful transport response can still contain an unusable payload, a truncated result, or no usage information. Validate the response schema, cap accepted output length, and store the request id beside the result. Emit structured events such as summary.requested, summary.completed, and summary.failed. Include tenant id, batch id, document id, attempt, status, and measured usage where available. Do not put ticket text in ordinary logs. Large payloads make the signal difficult to scan and create a privacy problem. A dashboard should show batches by status.

操作记录应该回答什么问题？有用的问题不仅是“API 是否返回了 200？”，而是“支持团队能否向租户解释这个结果和使用量？”成功的传输响应仍可能包含不可用的负载、截断的结果或缺失的使用信息。验证响应模式，限制接受的输出长度，并将请求 ID 存储在结果旁边。发出结构化事件，例如 summary.requested、summary.completed 和 summary.failed。在可用时包含租户 ID、批处理 ID、文档 ID、尝试次数、状态和测量使用量。不要将工单文本放入普通日志中。大型负载会使信号难以扫描，并引发隐私问题。仪表板应按状态显示批处理情况。