Node.js API Key Text Classification: JSON Validation Before Multi-Provider Gateway Failover

Node.js API Key 文本分类：多模型网关故障转移前的 JSON 验证

Short answer: For private knowledge-base tagging, compare a multi-provider LLM gateway by valid, policy-compliant classifications per unit of spend, not by the cheapest advertised token rate. One API key reduces credential and adapter work, but JSON mode is only a transport promise; your Node.js boundary still needs to parse, validate, reject, and selectively retry every answer. 简短回答：对于私有知识库的标签分类，评估多模型 LLM 网关的标准应是“单位支出下合规且有效的分类结果”，而非广告宣传的最低 Token 单价。虽然单一 API Key 可以减少凭证和适配器的工作量，但 JSON 模式仅仅是一种传输承诺；你的 Node.js 边界层仍然需要对每一个回答进行解析、验证、拒绝和选择性重试。

The decision rule is blunt: keep the gateway only if the same frozen evaluation set produces acceptable labels and schema-valid JSON across the model routes you will actually enable. Otherwise, use direct provider adapters and accept the extra config. 决策准则很直接：只有当同一套固定的评估集在所有你启用的模型路径上都能产生可接受的标签和符合 Schema 的 JSON 时，才保留网关。否则，请使用直接的提供商适配器，并接受额外的配置成本。

What changed the gateway choice? A private developer-tools knowledge base sounds like a small classification job. Give each document one primary tag, a confidence value, and a short reason. The awkward part is that a syntactically valid object can still be wrong: confidence may be a string, a tag may fall outside the approved taxonomy, or the model may classify instructions embedded in a document instead of classifying the document itself. JSON mode doesn’t settle any of those cases. 是什么改变了网关的选择？私有开发者工具知识库听起来是一个小型的分类任务。为每个文档分配一个主要标签、一个置信度值和一个简短理由。棘手之处在于，语法正确的对象也可能是错误的：置信度可能是字符串，标签可能超出了批准的分类体系，或者模型可能分类了文档中嵌入的指令，而不是文档本身。JSON 模式无法解决这些问题。

So I would benchmark the boundary, not the demo. The fixture set should contain ordinary docs, empty bodies, ambiguous release notes, code-heavy pages, and text that tries to redirect the classifier. Freeze the prompt, taxonomy, expected acceptance rules, and model identifiers for each run. Then record parse success, schema success, allowed-tag success, agreement with reviewed labels, latency, and total billed usage. 因此，我建议基准测试边界层，而不是演示程序。测试集应包含普通文档、空内容、模棱两可的发布说明、代码密集型页面以及试图误导分类器的文本。在每次运行中固定提示词（Prompt）、分类体系、预期的验收规则和模型标识符。然后记录解析成功率、Schema 成功率、允许标签的成功率、与人工审核标签的一致性、延迟以及总计费使用量。

I’m not sure which route wins on a particular corpus; nobody can know without those reviewed labels and current billing data. Your mileage may vary. This is where “cheapest routing” gets slippery. A low-cost response that fails validation and consumes a retry isn’t cheap. A fallback that returns valid JSON but changes the label is not recovery either — it is an observable classification decision that needs its own test. Short version: benchmark accepted work. 我不确定哪种路径在特定语料库上表现最好；没有那些审核过的标签和当前的计费数据，没人能知道。实际效果可能因人而异。这就是“最便宜路由”容易产生误导的地方。一个低成本但验证失败并消耗重试次数的响应并不便宜。一个返回了有效 JSON 但改变了标签的后备方案也不是真正的恢复——它是一个可观测的分类决策，需要进行单独的测试。简而言之：请对“被接受的工作”进行基准测试。

The provider boundary also changes what the comparison means. OpenAI’s function-calling guide documents structured function arguments. Claude and Gemini expose their own native model interfaces, so a direct multi-provider build needs provider-specific adapters around a common application contract. A gateway such as OpenRouter offers a common access layer across models, which can reduce that glue, but normalization cannot define your private taxonomy or decide whether a label is correct. Those are application responsibilities. 提供商边界也改变了比较的意义。OpenAI 的函数调用指南记录了结构化的函数参数。Claude 和 Gemini 暴露了各自的原生模型接口，因此直接构建多提供商方案需要在通用应用契约周围编写特定于提供商的适配器。像 OpenRouter 这样的网关提供了跨模型的通用访问层，可以减少这些胶水代码，但标准化无法定义你的私有分类体系，也无法决定标签是否正确。这些是应用程序的责任。

Connection Useful comparison boundary

连接 有用的比较边界

Engineering cost to retain	OpenAI direct	Native function and schema behavior	A direct-provider adapter and credential
保留的工程成本	OpenAI 直接调用	原生函数和 Schema 行为	直接提供商适配器和凭证

Claude direct	Native request and tool contract	A second adapter and credential
Claude 直接调用	原生请求和工具契约	第二个适配器和凭证

Gemini direct	Native request and schema contract	A third adapter and credential
Gemini 直接调用	原生请求和 Schema 契约	第三个适配器和凭证

OpenRouter gateway	One access layer across model routes	Gateway semantics plus output validation
OpenRouter 网关	跨模型路由的统一访问层	网关语义加输出验证

How should one API key route multiple LLM providers for JSON text classification fallback? Put routing after validation, not around it. The application sends one internal request shape to an adapter. The adapter returns unknown data. A validator converts that untrusted value into the only result the rest of the CLI is allowed to see. This keeps gateway convenience out of business logic and gives direct connections the same contract. 如何使用一个 API Key 为 JSON 文本分类故障转移路由多个 LLM 提供商？将路由放在验证之后，而不是围绕它。应用程序向适配器发送一种内部请求格式。适配器返回未知数据。验证器将该不可信的值转换为 CLI 其余部分唯一可见的结果。这使得网关的便利性不会干扰业务逻辑，并为直接连接提供了相同的契约。

Here is the smallest implementation I would be willing to ship. It uses no SDK-specific types and no provider routes, so swapping the transport doesn’t leak through the codebase. 这是我愿意发布的最小化实现。它不使用任何 SDK 特定的类型，也不使用提供商路由，因此更换传输方式不会导致代码库泄露。

const tags = ["api", "cli", "deployment", "testing"] as const;
type Tag = (typeof tags)[number];
type Classification = { tag: Tag; confidence: number; reason: string; };
type ModelTarget = "primary" | "fallback";
type Invoke = (target: ModelTarget, input: string) => Promise<unknown>;

function isRecord(value: unknown): value is Record<string, unknown> {
  return typeof value === "object" && value !== null && !Array.isArray(value);
}

function parseClassification(value: unknown): Classification {
  if (!isRecord(value)) throw new Error("CLASSIFY_NOT_OBJECT");
  const keys = Object.keys(value).sort();
  const expected = ["confidence", "reason", "tag"];
  if (keys.join(",") !== expected.join(",")) {
    throw new Error("CLASSIFY_WRONG_KEYS");
  }
  if (!tags.includes(value.tag as Tag)) {
    throw new Error("CLASSIFY_UNKNOWN_TAG");
  }
  if (typeof value.confidence !== "number" || value.confidence < 0 || value.confidence > 1) {
    throw new Error("CLASSIFY_BAD_CONFIDENCE");
  }
  if (typeof value.reason !== "string" || value.reason.length === 0) {
    throw new Error("CLASSIFY_BAD_REASON");
  }
  return { tag: value.tag as Tag, confidence: value.confidence, reason: value.reason };
}

export async function classify(input: string, invoke: Invoke): Promise<Classification> {
  try {
    return parseClassification(await invoke("primary", input));
  } catch (error) {
    if (!(error instanceof Error) || !error.message.startsWith("CLASSIFY_")) {
      throw error;
    }
    return parseClassification(await invoke("fallback", input));
  }
}

That error prefix is deliberate. A broad catch turns programmer mistakes, authentication failures, and policy rejections into surprise model traffic. Don’t do that. The example retries only failures created by the classification boundary, and it validates the fallback with the exact same function. There is another catch: retrying malformed output may be reasonable, while retrying a valid but low-confidence output silently changes policy. Keep those paths separate. If confidence is valid yet below your review threshold, enqueue human review or apply a documented business rule; don’t pretend a second model is an exception handler. 那个错误前缀是刻意为之的。宽泛的捕获会将程序员的错误、身份验证失败和策略拒绝转化为意外的模型流量。不要这样做。该示例仅重试由分类边界产生的失败，并使用完全相同的函数验证后备方案。还有一点需要注意：重试格式错误的输出可能是合理的，但重试一个有效但置信度低的输出会悄悄改变策略。请将这些路径分开。如果置信度有效但低于你的审核阈值，请将其加入人工审核队列或应用已记录的业务规则；不要假装第二个模型是异常处理程序。

The eval catches what JSON mode misses: The useful test artifact is a versioned set of inputs and reviewed labels. Run every candidate route against the same set with temperature and other controllable parameters held constant, then save the raw response, validated response, route identifier, prompt version, etc. 评估能捕捉到 JSON 模式遗漏的内容：有用的测试产物是一套带版本控制的输入和审核过的标签。在保持温度（Temperature）和其他可控参数不变的情况下，对同一集合运行每个候选路由，然后保存原始响应、验证后的响应、路由标识符、提示词版本等。