Context Engineering Is Changing. Here’s What It Means for Data Scientists

Context Engineering Is Changing. Here’s What It Means for Data Scientists

上下文工程正在发生变革:这对数据科学家意味着什么?

There are so many positive sides that come with using systems like Claude; all the repetitive, routine coding gets automated, researching is quicker, and debugging becomes easier. In general, for people in software, productivity goes up and allows teams to ship more. 使用 Claude 这类系统有很多积极的一面:所有重复性的日常编码工作都能实现自动化,研究效率更高,调试也变得更简单。总的来说,对于软件从业者而言,生产力得到了提升,使团队能够交付更多成果。

However, the slight drawback of using tools like Claude is that you have to stay on track with these agents. This is not only to avoid being left behind and using outdated technology. This is also because new models are trained to work slightly differently than their predecessors. 然而,使用 Claude 这类工具的一个小缺点是,你必须紧跟这些智能体的更新步伐。这不仅是为了避免落后于时代或使用过时的技术,还因为新模型的训练方式与前代产品略有不同。

Ultimately, this means that you might find yourself asking, “What’s wrong with Claude today? Why is it not working properly?” not realizing that you are not using Claude correctly according to the new expected procedures. This is the reason why companies like Anthropic show not only that new models are objectively better than older ones according to some benchmarks, but they also communicate to users how they should change the way they interact with them. 归根结底,这意味着你可能会问:“Claude 今天怎么了?为什么它运行不正常?”却没意识到你并没有按照新的预期流程正确使用它。这就是为什么像 Anthropic 这样的公司不仅通过基准测试证明新模型客观上优于旧模型,还会告知用户应如何改变与它们的交互方式。

Less than a month ago, Anthropic released “the new rules of Context Engineering for Claude 5 generation models”. The article explains in detail how the models are changing, and that prompt engineering is not the main character anymore, leaving space for context engineering, which is the collection of all the sources that LLMs use to answer your question/complete your ask. 不到一个月前,Anthropic 发布了“Claude 5 代模型上下文工程新规则”。该文章详细解释了模型正在发生的变化,并指出提示词工程(Prompt Engineering)已不再是主角,取而代之的是上下文工程(Context Engineering)——即大语言模型用于回答问题或完成任务的所有来源集合。

In particular, this article talks about us, Data Scientists, and tries to answer this question: “How do context engineering’s new rules change our day-to-day work?” But before we do that… what is even “context engineering”? 特别是,本文专门探讨了我们数据科学家群体,并试图回答这个问题:“上下文工程的新规则如何改变我们的日常工作?”但在深入探讨之前……究竟什么是“上下文工程”?

0. What is Context Engineering?

0. 什么是上下文工程?

You can think of two big ways to use Anthropic models: API Calls and Claude Code. 你可以将使用 Anthropic 模型的方式大致分为两大类:API 调用和 Claude Code。

API calls are one-shot processing of the input. Your text (the input sequence string) gets processed by the LLM, which generates the output sequence string. Because you don’t own the LLM, the processing happens online, and you pay for input and output tokens. That’s it. API 调用是对输入的一次性处理。你的文本(输入序列字符串)由大语言模型处理,并生成输出序列字符串。由于你并不拥有该模型,处理过程是在线进行的,你需要为输入和输出的 Token 付费。仅此而已。

Systems like Claude Code (or Codex) work agentically. This means that the model is still being used and called through API calls, but it is doing much more than just reading the input. A multitude of API calls are executed, and every API call is responsible for something different (e.g., reasoning, coding, generating the output etc). What you see at the end is the final result of all the processing that went on through different API calls. 像 Claude Code(或 Codex)这样的系统是以智能体(Agentic)方式工作的。这意味着模型仍然通过 API 调用来使用,但它所做的远不止读取输入。系统会执行大量的 API 调用,每个调用负责不同的任务(例如推理、编码、生成输出等)。你最终看到的是通过不同 API 调用处理后的最终结果。

For cases where you need (1.), prompt engineering, which is the art of crafting the perfect input information, is extremely important: you are doing a specific task, all the information to execute the task is in the prompt, and you expect a reliable and verifiable answer. 在需要(1)的情况下,提示词工程(即精心构建完美输入信息的艺术)至关重要:你正在执行一项特定任务,所有执行任务所需的信息都在提示词中,并且你期望得到一个可靠且可验证的答案。

When you use an agent (2.), prompt engineering is much less the main character. This is because agents are meant to be intuitive, easy to use, and generally more powerful as they leverage massive internal systems. 当你使用智能体(2)时,提示词工程就不再是主角了。这是因为智能体旨在直观、易用,并且由于它们利用了庞大的内部系统,通常功能更强大。

However, this doesn’t mean that Claude can read your mind :) All the information that is still necessary for Claude to do a good job should be in a set of documents, instruction files, and settings that are defined as “context.” This context needs to be “engineered” in order to be used optimally by the agentic system. Anthropic gives us guidelines to do so. Now, let’s review these guidelines using our Data Science detective hat. 然而,这并不意味着 Claude 能读懂你的心思 :) Claude 要出色完成工作所需的所有信息,都应包含在一组文档、指令文件和设置中,这些统称为“上下文”。为了让智能体系统实现最优使用,必须对这些上下文进行“工程化”。Anthropic 为我们提供了相关指南。现在,让我们戴上数据科学家的侦探帽,来审视这些指南。

1. “Trust me, bro”

1. “相信我,兄弟”

The first point of the article talks about “CLAUDE.md” which is the file that gets created by Claude Code when you initialize a folder. About “CLAUDE.md”, Anthropic team basically says: stop patronizing Claude and giving it too many instructions; you are only confusing it. The point that they make is that new models can fill the gap, and by adding an excessive amount of info, you are just adding the probability of this info being incompatible with each other. 文章的第一点谈到了“CLAUDE.md”,这是当你初始化一个文件夹时,Claude Code 会自动创建的文件。关于“CLAUDE.md”,Anthropic 团队的核心观点是:别再对 Claude 指手画脚,给它过多的指令了;你只会让它感到困惑。他们指出,新模型已经具备了填补空白的能力,添加过量的信息只会增加信息之间相互冲突的可能性。

This means you can write in “CLAUDE.md” whether the folder is an EDA folder, a research folder, or a “code to be delivered in production” folder. Then, Claude will be able to understand that it needs to be extra careful when implementing a feature if it is to be delivered in production, and you will probably use .py files rather than notebooks, while the opposite will be true in an EDA folder. 这意味着你可以在“CLAUDE.md”中注明该文件夹是用于 EDA(探索性数据分析)、研究,还是“待交付生产的代码”。这样,Claude 就能理解,如果是生产环境代码,它在实现功能时需要格外谨慎,你可能更倾向于使用 .py 文件而非 Notebook;而在 EDA 文件夹中则恰恰相反。

2. “No spoilers!“

2. “别剧透!”

Claude is designed to use skills. You can think of a skill as something that saves you time: when you are doing the same thing over and over again, you set this information in a given file, and you redirect your conversation to that file rather than explaining from scratch and repeating yourself. Claude 被设计为能够使用“技能”(Skills)。你可以将技能视为一种节省时间的方式:当你反复做同一件事时,将相关信息设置在特定文件中,并在对话中引导它参考该文件,而不是从头开始解释并重复自己。

For example, you might have a notebook.md skill where you instruct Claude on how to work with your notebooks, what colors you prefer for the plots, what libraries to use to compute the statistics, etc. 例如,你可以创建一个 notebook.md 技能文件,指导 Claude 如何处理你的 Notebook、你偏好的绘图颜色、用于计算统计数据的库等。

Anthropic is basically telling us: “If you put everything in one long skill, Claude will not know what is actually relevant in your ask.” In other words, they are asking us to be specific. Let’s watch this from a Data Science perspective. We have many different tasks to take care of, and they are different from each other. Sometimes we load the data, sometimes we just explore it, sometimes we set up Databricks and spin up a cluster, sometimes we train a model. Each microtask should create a skill, which is ultimately wrapped in another macro skill. At the end of our design, we will build a taxonomy, i.e., a tree of skills. Anthropic 实际上是在告诉我们:“如果你把所有东西都塞进一个长技能文件中,Claude 就无法分辨哪些信息与你的请求真正相关。”换句话说,他们要求我们具体化。让我们从数据科学的角度来看:我们需要处理许多不同的任务,且它们各不相同。有时我们需要加载数据,有时只是进行探索,有时需要配置 Databricks 并启动集群,有时则是在训练模型。每个微任务都应创建一个技能,最终将其封装在另一个宏观技能中。在设计结束时,我们将构建一个分类体系,即一个技能树。