superlinked / sie
superlinked / sie
SIE: Superlinked Inference Engine Self-hosted inference for agents. Every open model your agents call, served from one cluster in your cloud. SIE：Superlinked 推理引擎 面向智能体的自托管推理引擎。让你的智能体调用的每一个开源模型，都由云端的一个集群统一提供服务。
About SIE is an open-source inference engine that runs the models behind every agent task through one API: search and retrieval, document-to-markdown conversion, structured output, content safety, and the agent loop itself. It replaces the patchwork of a separate model server per task with one system that serves 100+ models, loading each on demand. 关于 SIE SIE 是一个开源推理引擎，通过单一 API 运行智能体任务背后的所有模型：包括搜索与检索、文档转 Markdown、结构化输出、内容安全以及智能体循环本身。它取代了以往为每个任务配置独立模型服务器的零散方案，通过一个系统即可服务 100 多个模型，并支持按需加载。
- OpenAI-compatible API for drop-in migration:
/v1/embeddings,/v1/chat/completions,/v1/completions,/v1/responses - Pre-configured model catalog: Stella, SPLADE, Qwen3, GLiNER, SigLIP, and more; embedding and retrieval models benchmarked on MTEB
- Serves multiple models simultaneously with on-demand loading and LRU eviction
- Ships Kubernetes and Helm deployment configs for the load-balancing gateway, KEDA autoscaling, and Grafana dashboards
- Integrates with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB
- 兼容 OpenAI API，可直接迁移：
/v1/embeddings、/v1/chat/completions、/v1/completions、/v1/responses - 预配置模型目录：包含 Stella、SPLADE、Qwen3、GLiNER、SigLIP 等；嵌入和检索模型均已在 MTEB 上进行基准测试
- 支持多模型并发服务，具备按需加载和 LRU（最近最少使用）淘汰机制
- 提供 Kubernetes 和 Helm 部署配置，支持负载均衡网关、KEDA 自动扩缩容以及 Grafana 仪表盘
- 集成 LangChain、LlamaIndex、Haystack、DSPy、CrewAI、Chroma、Qdrant、Weaviate 和 LanceDB
Development
Install mise, then bootstrap the versioned Python, Rust, Node.js, and Helm toolchains from the repository root: ./tools/init.sh
The common development checks and local server are available as mise tasks: mise run test, mise run lint, mise run typecheck, mise run serve, mise run rust-check, mise run rust-test, mise run gateway-test, mise run server-sidecar-test
开发指南
安装 mise，然后从仓库根目录引导版本化的 Python、Rust、Node.js 和 Helm 工具链：./tools/init.sh
常见的开发检查和本地服务器任务可通过 mise 运行：mise run test、mise run lint、mise run typecheck、mise run serve、mise run rust-check、mise run rust-test、mise run gateway-test、mise run server-sidecar-test
Tasks
One SIE cluster runs the inference behind a whole agent. Each task is a handful of swappable models; browse packages/sie_server/models/ for the full set.
任务
一个 SIE 集群即可运行整个智能体背后的推理任务。每个任务包含一组可替换的模型；查看 packages/sie_server/models/ 获取完整列表。
| Task | What it does | Models |
|---|---|---|
| Search | Embed, match, and rerank to retrieve the right context. | bge-m3, splade-v3, colbertv2, qwen3-reranker |
| Document to markdown | PDFs, Office files, and scans become clean markdown. | lightonocr, glm-ocr, mineru, paddleocr-vl, docling |
| Structured output | Schema-valid JSON, extracted or generated. | gliner2, nuner-zero, qwen3.6-27b |
| Guard content | A safety verdict with a probability you threshold. | granite-guardian-2b |
| Run the agent loop | Plan steps and call tools with an open LLM, streaming included. | qwen3.6-27b |
| 任务 | 功能描述 | 模型 |
|---|---|---|
| 搜索 | 嵌入、匹配并重排序以检索正确的上下文。 | bge-m3, splade-v3, colbertv2, qwen3-reranker |
| 文档转 Markdown | 将 PDF、Office 文件和扫描件转换为整洁的 Markdown。 | lightonocr, glm-ocr, mineru, paddleocr-vl, docling |
| 结构化输出 | 生成或提取符合 Schema 的 JSON。 | gliner2, nuner-zero, qwen3.6-27b |
| 内容防护 | 提供带有概率阈值的安全判定。 | granite-guardian-2b |
| 运行智能体循环 | 使用开源大模型规划步骤并调用工具，支持流式传输。 | qwen3.6-27b |
Quickstart
- Start the server
(Docker commands omitted for brevity, see original documentation for specific environment setups)
The server speaks the OpenAI API out of the box, embeddings and generation alike. Your first call needs nothing but curl:
curl http://localhost:8080/v1/embeddings -H 'Content-Type: application/json' -d '{"model": "sentence-transformers/all-MiniLM-L6-v2", "input": "Hello world"}'快速开始 - 启动服务器
（此处省略 Docker 命令，请参考原始文档获取特定环境配置）
服务器开箱即用支持 OpenAI API，无论是嵌入还是生成任务均可。首次调用只需使用 curl：
curl http://localhost:8080/v1/embeddings -H 'Content-Type: application/json' -d '{"model": "sentence-transformers/all-MiniLM-L6-v2", "input": "Hello world"}'
- Install the SDK
pip install sie-sdk(Python)npm install @superlinked/sie-sdk(TypeScript) 2. 安装 SDKpip install sie-sdk(Python)npm install @superlinked/sie-sdk(TypeScript)
- Generate embeddings, rerank, and extract entities
(Code examples for
SIEClientusage provided in original text) 3. 生成嵌入、重排序并提取实体 （代码示例见原文中SIEClient的使用方法）
Production The same code works against a production cluster. SIE ships a load-balancing gateway and Kubernetes deployment surface, including Helm charts, KEDA autoscaling (scale to zero), and Grafana dashboards. 生产环境 相同的代码可直接用于生产集群。SIE 提供了负载均衡网关和 Kubernetes 部署方案，包括 Helm Charts、KEDA 自动扩缩容（支持缩容至零）以及 Grafana 仪表盘。