Kimi K3’s 1M Token Context Window vs. RAG: Cost, Latency and Answer Quality
Kimi K3’s 1M Token Context Window vs. RAG: Cost, Latency and Answer Quality
Kimi K3 的 100 万 Token 上下文窗口与 RAG：成本、延迟及回答质量对比
A controlled comparison of a top-5 RAG pipeline and a full 127,000 token prompt on the same 12 questions, same system prompt and same model. Graded blind on correctness, completeness and grounding. Sarah Schürch Aug 19, 2026 23 min read Share Image from Raj Rana auf Unsplash
这是一项受控对比实验，针对相同的 12 个问题、相同的系统提示词（System Prompt）和相同的模型，对比了“前 5 检索结果的 RAG 流水线”与“完整的 127,000 token 提示词”。评分过程采用盲测，评估维度包括准确性、完整性和依据性。作者：Sarah Schürch，2026 年 8 月 19 日，阅读时长 23 分钟。图片来源：Raj Rana (Unsplash)。
When Kimi K3 came out in July with a context window of one million tokens, I asked myself the same question a lot of people probably did: can I skip RAG now? The math looks tempting. Everything A selection of what I have published over the last two years adds up to 127,068 tokens. That is twelve percent of the window. So I could simply dump all of it into the prompt instead of building chunks, computing embeddings and worrying about retrieval quality.
当 Kimi K3 在 7 月发布并带来 100 万 token 的上下文窗口时，我和许多人一样问了自己同一个问题：现在我可以跳过 RAG（检索增强生成）了吗？从数学角度看，这很有吸引力。我过去两年发表的文章总计 127,068 个 token，仅占该窗口的 12%。因此，我完全可以直接将所有内容塞进提示词中，而无需再进行分块、计算嵌入（embeddings）以及担心检索质量。
But “it fits” and “it works better” are two different things. That is exactly what I wanted to measure in a small experiment. In this article I show you what happens when the same twelve questions are answered once through a RAG setup and once through the full corpus in the context window (I call it the long_context setup here). I also show you what went wrong along the way, which turned out to be almost more instructive than the experiment itself.
但“装得下”和“效果更好”是两码事。这正是我想要通过一个小实验来验证的内容。在本文中，我将展示当同一个问题通过 RAG 设置和“长上下文设置”（long_context setup，即直接放入完整语料库）分别回答时会发生什么。我还会展示过程中出现的问题，事实证明，这些问题比实验本身更具启发性。
1 — Why the question looks different now
1 — 为什么现在这个问题变得不同了
RAG (Retrieval Augmented Generation) came about as an answer to a technical limitation: a model could only see a few thousand tokens at a time. If you had more material than that, you had to pick out the relevant passages first and pass only those on. That works well, but it adds another component to our system that we have to maintain, tune and debug. This is where Kimi K3 comes in. The model by Moonshot AI offers a window of one million tokens. There are also other models with similarly large context windows. This removes one of the main reasons why many people have used RAG.
RAG（检索增强生成）的出现是为了解决一个技术限制：模型一次只能处理几千个 token。如果你拥有的资料超过这个限制，就必须先挑选出相关段落并仅传递这些内容。这虽然有效，但给系统增加了一个需要维护、调优和调试的组件。Kimi K3 的出现改变了这一点。月之暗面（Moonshot AI）提供的模型拥有 100 万 token 的窗口，其他模型也具备类似的超长上下文窗口。这消除了许多人使用 RAG 的主要原因之一。
Other advantages still cited in favour of RAG include: It’s cheaper. It’s faster. Latency is lower. It gives you traceable sources. You can read more about this topic in the paper by Google DeepMind. Those three are what I wanted to test, on a corpus where I can judge every answer myself, because I wrote it.
人们支持 RAG 的其他理由还包括：成本更低、速度更快、延迟更低，且能提供可追溯的来源。你可以在 Google DeepMind 的论文中阅读更多相关内容。我正是想在自己撰写的语料库上测试这三点，因为我可以亲自评估每一个回答。
2 — The setup: One corpus, two paths
2 — 设置：一个语料库，两条路径
The corpus consists of 32 files holding 33 of my own articles from Medium and Towards Data Science. That makes a total of 127,068 tokens. I counted with tiktoken and cl100k_base, so not with Kimi’s own tokenizer. What Moonshot billed later was 127,346 input tokens per request, and that also includes the system instruction and the question. So the estimate was close enough to plan with.
语料库包含 32 个文件，涵盖了我自己在 Medium 和 Towards Data Science 上发表的 33 篇文章，总计 127,068 个 token。我使用 tiktoken 和 cl100k_base 进行计数，而非 Kimi 自己的分词器。Moonshot 最终计费为每次请求 127,346 个输入 token，这包含了系统指令和问题，因此我的预估值足够用于规划。
The RAG path splits the articles into 788 chunks of 900 characters with 150 characters of overlap, embeds them with all-MiniLM-L6-v2 and sends the five most similar chunks to the model together with the question. That comes to roughly 1,200 tokens per request.
RAG 路径将文章拆分为 788 个块（每个块 900 字符，重叠 150 字符），使用 all-MiniLM-L6-v2 进行嵌入，并将最相似的 5 个块与问题一起发送给模型。每次请求大约消耗 1,200 个 token。
The long_context path sends all 32 articles along with every single question. That is 127,346 tokens per request, so about a hundred times as much. One detail decides the cost: the corpus always comes before the question in the prompt. Prefix caching only works as long as the beginning of the message stays identical character for character. If the question came first, every single call would be a cache miss and the whole experiment several times more expensive.
“长上下文”路径则将所有 32 篇文章与每个问题一起发送，每次请求 127,346 个 token，大约是 RAG 的 100 倍。一个细节决定了成本：语料库必须始终放在提示词中问题的前面。前缀缓存（Prefix caching）只有在消息开头字符完全一致时才有效。如果问题放在前面，每次调用都会导致缓存失效，整个实验的成本将高出数倍。