How We Built a 99.9% Uptime Multi-Model AI Router (Claude -> GPT-4o -> DeepSeek) in n8n Without SaaS Middleware

如何在 n8n 中构建 99.9% 在线率的多模型 AI 路由（Claude -> GPT-4o -> DeepSeek），且无需 SaaS 中间件

When running mission-critical LLM pipelines in production, relying on a single AI provider’s API endpoint is an operational liability. Between Anthropic’s intermittent 529 overload errors, OpenAI’s sporadic rate limits (429), and vendor-specific latency spikes, hardcoded single-model integrations inevitably cause dropped workflows and frustrated users. Many teams default to commercial AI gateway services charging $20–$100+/month just for API routing and proxying. In this architectural walkthrough, we explain how we designed and deployed AgentFlow OS—a completely self-hosted, multi-model failover engine built on n8n with zero external database dependencies.

在生产环境中运行关键任务的 LLM 流水线时，依赖单一 AI 提供商的 API 接口是一种运营风险。Anthropic 间歇性的 529 过载错误、OpenAI 零星的速率限制（429）以及特定供应商的延迟峰值，使得硬编码的单模型集成不可避免地会导致工作流中断和用户不满。许多团队默认选择商业 AI 网关服务，仅为了 API 路由和代理功能每月支付 20 到 100 美元以上。在本架构指南中，我们将解释如何设计并部署 AgentFlow OS——一个完全自托管、基于 n8n 构建且无需外部数据库依赖的多模型故障转移引擎。

1. The Core Architecture: Cascading Resilience

1. 核心架构：级联弹性

The primary design principle is graceful degradation with deterministic schema validation: 核心设计原则是基于确定性模式验证的优雅降级：

[ Incoming Webhook / Client Request ]
│
▼
[ Payload Sanitizer ]
│
▼
┌──► [ Primary Model: Claude 3.5 Sonnet ]
│ │ Success? ├─► [OK: 200] ──► [ Schema Validator ] ──► [ Response / DB ]
│ ▼
│ [ Fail: 429/500/529 ]
│ │
│ ├──► [ Secondary Model: GPT-4o / Mini ]
│ │ Success? ├─► [OK: 200] ──► [ Schema Validator ] ──► [ Response / DB ]
│ ▼
│ [ Fail: 429/500/Timeout ]
│ │
│ └──► [ Tertiary Model: DeepSeek V3 ]
│ ├─► [OK: 200] ──► [ Schema Validator ] ──► [ Response / DB ]
▼
[ All Failed ] ──► [ Telegram / Ops Alert & Dead-Letter Queue ]

Why this specific cascade? 为什么选择这种特定的级联方式？

• Primary (Claude 3.5 Sonnet): Unmatched reasoning, nuanced code generation, and complex instruction following. 主模型 (Claude 3.5 Sonnet)： 无与伦比的推理能力、细腻的代码生成以及对复杂指令的遵循能力。
• Secondary (OpenAI GPT-4o): Extremely fast token throughput, high concurrent rate limits, reliable fallback. 次级模型 (OpenAI GPT-4o)： 极快的 Token 吞吐量、高并发速率限制，可靠的备选方案。
• Tertiary (DeepSeek V3): Cost-effective emergency compute layer when Western providers experience localized outages or heavy load. 三级模型 (DeepSeek V3)： 当西方供应商遇到局部中断或高负载时，提供高性价比的紧急计算层。

2. Implementing the Failover Logic in n8n

2. 在 n8n 中实现故障转移逻辑

In standard n8n workflows, an HTTP node error terminates workflow execution. To implement true resilience: 在标准的 n8n 工作流中，HTTP 节点错误会终止工作流执行。为了实现真正的弹性：

A. Disable “Stop on Error” Under node settings for each HTTP Request / AI Model node: Set OnError to “Continue Regular Output” or route to an alternate error branch. Inspect {{ $json.error }} or HTTP status code in the subsequent Switch node.

A. 禁用“出错时停止 (Stop on Error)” 在每个 HTTP 请求/AI 模型节点的设置中：将 OnError 设置为“继续常规输出 (Continue Regular Output)”或路由到备用错误分支。在随后的 Switch 节点中检查 {{ $json.error }} 或 HTTP 状态码。

B. Standardized Response Normalization Different model APIs return different JSON envelope formats. We pass outputs through a lightweight JavaScript Code Node to normalize them into a uniform internal contract.

B. 标准化响应归一化 不同的模型 API 返回不同的 JSON 封装格式。我们通过一个轻量级的 JavaScript 代码节点处理输出，将其归一化为统一的内部契约。

// Normalization Code Node in n8n
const item = $input.first().json;
let content = "";
let modelUsed = "";
let tokensUsed = { prompt: 0, completion: 0 };

if (item.content && Array.isArray(item.content)) { // Anthropic Claude format
    content = item.content.map(c => c.text).join("");
    modelUsed = item.model || "claude-3-5-sonnet";
    tokensUsed = { prompt: item.usage?.input_tokens || 0, completion: item.usage?.output_tokens || 0 };
} else if (item.choices && item.choices[0]?.message) { // OpenAI / DeepSeek format
    content = item.choices[0].message.content;
    modelUsed = item.model || "gpt-4o";
    tokensUsed = { prompt: item.usage?.prompt_tokens || 0, completion: item.usage?.completion_tokens || 0 };
} else {
    throw new Error("Unrecognized API payload response structure");
}

return { json: { success: true, content: content.trim(), model_provider: modelUsed, usage: tokensUsed, timestamp: new Date().toISOString() } };

3. Strict Schema Validation: Enforcing Zero-Hallucination JSON

3. 严格的模式验证：强制实现零幻觉 JSON

A common pitfall with fallback models is output drift: Model B might omit a required JSON field that Model A reliably populated. To protect downstream services, we place a JSON Schema Validator immediately after response normalization.

使用备用模型时的一个常见陷阱是输出漂移：模型 B 可能会遗漏模型 A 可以可靠填充的必要 JSON 字段。为了保护下游服务（CRM、SQL 数据库、电子邮件自动化），我们在响应归一化后立即放置了一个 JSON 模式验证器。

(Code snippet omitted for brevity, but follows standard jsonschema validation logic) (代码片段略，遵循标准的 jsonschema 验证逻辑)

If validation fails, the workflow does not silently pass malformed data; it triggers a 1-shot self-repair prompt or cascades to the next model. 如果验证失败，工作流不会静默传递格式错误的数据；它会触发一次性自修复提示，或级联到下一个模型。

4. Operational Telemetry & Alerting

4. 运营遥测与警报

When all providers in the cascade fail or rate-limits are completely exhausted, the system automatically dispatches an emergency notification to a dedicated Telegram Ops channel with the request payload context and stack trace, while archiving the payload into a local JSON dead-letter directory for offline replay.

当级联中的所有提供商都失败或速率限制完全耗尽时，系统会自动向指定的 Telegram 运维频道发送紧急通知，包含请求负载上下文和堆栈跟踪，同时将负载归档到本地 JSON 死信目录以便离线重试。

5. Key Takeaways & Production Assets

5. 关键要点与生产资产

• Zero Middleware Tax: By orchestrating directly inside n8n, you eliminate third-party SaaS proxy fees and maintain full data privacy. 零中间件税： 通过在 n8n 内部直接编排，消除了第三方 SaaS 代理费用，并保持了完整的数据隐私。
• Resilience by Design: Multi-tier failover guarantees your client-facing applications never show blank screens or generic 500 errors. 设计弹性： 多层故障转移确保面向客户的应用程序永远不会显示空白屏幕或通用的 500 错误。
• Deterministic Contracts: Always validate AI JSON responses against formal schemas before touching operational databases. 确定性契约： 在触及运营数据库之前，始终根据正式模式验证 AI 的 JSON 响应。