I Thought Loading Data Was the Finish Line. It Was the Starting Point.
I Thought Loading Data Was the Finish Line. It Was the Starting Point.
我曾以为加载数据就是终点，其实那只是起点
Data Engineering: Building my first dbt models and learning what “analysis-ready” data actually means 数据工程：构建我的首个 dbt 模型，并理解什么是真正的“分析就绪”数据
Ibrahim Salami | Aug 9, 2026 | 11 min read Ibrahim Salami | 2026年8月9日 | 11分钟阅读
When I started this journey, I gave myself a 12-month roadmap to go from data analyst to data engineer. I’m only about two months into it. In that short stretch I’ve already built two ETL pipelines from scratch, the first one pulling GitHub repo data into SQLite, the second one pulling RSS articles into PostgreSQL with Docker and Kestra handling the orchestration. I wrote about scheduling that second pipeline to run automatically every hour, and at the time, that felt like a real milestone. The data was flowing in on its own, no manual runs, no me remembering to trigger anything. 当我开启这段旅程时，我为自己制定了从数据分析师转型为数据工程师的12个月路线图。目前我才进行了大约两个月。在这短短的时间里，我已经从零开始构建了两个 ETL 流水线：第一个是将 GitHub 仓库数据拉取到 SQLite，第二个是通过 Docker 和 Kestra 编排，将 RSS 文章拉取到 PostgreSQL。我曾写过关于如何调度第二个流水线使其每小时自动运行的文章，当时我觉得那是一个真正的里程碑。数据在自动流入，无需手动运行，也不用我费心去触发任何东西。
But somewhere between writing that article and starting this one, I ran a query on my own data and realized something. I couldn’t sort my articles by date properly. I couldn’t tell which blogs were publishing the most. The data had been sitting in Postgres for weeks, technically “loaded,” and I hadn’t actually looked at it closely until I needed it for something. Turns out I’d built two pipelines and skipped the part that makes the data useful. Extract, load, and then nothing. No transformation, no modeling, no real structure past “it’s in a table now.” 但在写完那篇文章到开始写这一篇的某个时间点，我对自己数据库中的数据运行了一个查询，然后意识到了一些问题。我无法按日期正确地对文章进行排序，也无法判断哪些博客发布的内容最多。这些数据在 Postgres 中已经存放了数周，从技术上讲是“已加载”的，但在我真正需要使用它们之前，我从未仔细审视过。事实证明，我构建了两个流水线，却跳过了让数据变得有用的关键环节。只有提取和加载，然后就没有了。没有转换，没有建模，除了“它现在在表里”之外，没有任何真正的结构。
This article is about fixing that. I finally sat down and learned dbt, and in the process learned what “analysis ready” actually means, because it turns out loading data and having usable data are two very different things. 这篇文章就是为了解决这个问题。我终于静下心来学习了 dbt，并在过程中理解了什么是真正的“分析就绪”，因为事实证明，加载数据和拥有可用的数据是两码事。
The Data Was Loaded. It Just Wasn’t Usable.
数据加载了，但它不可用。
Here’s what my articles table actually looked like once I stopped and paid attention to it. The schema itself was simple, honestly about as simple as a table can get:
当我停下来仔细观察时，我的 articles 表实际上是这样的。架构本身很简单，老实说，这几乎是你能见到的最简单的表结构了：
CREATE TABLE IF NOT EXISTS articles (
id TEXT PRIMARY KEY,
title TEXT NOT NULL,
link TEXT NOT NULL,
summary TEXT,
published TEXT
);Notice that last column. published is a TEXT field. Not a timestamp, not a date, just a plain string that happened to look like a date if you squinted at it. When I queried the ten most recent articles, this is what came back:
注意最后一列。published 是一个 TEXT 字段。它不是时间戳，也不是日期，只是一个看起来像日期的普通字符串。当我查询最近的十篇文章时，返回的结果如下：
| title | published |
|---|---|
| Django Weblog: Last Call 2026 Django Developer Survey | Wed, 08 Jul 2026 19:31:21 +0000 |
| Mike Driscoll: New Book Release: Python Typing | Wed, 08 Jul 2026 18:46:18 +0000 |
That looks fine at a glance. It’s readable. But try to actually do anything with it. Want the articles from the last 7 days? You can’t filter on that without casting it first, every single time, in every single query. Want to sort chronologically and trust the order? Text sorting and date sorting aren’t the same thing, and depending on the format, they can quietly disagree with each other. 乍一看这没问题，它是可读的。但试着对它做点什么吧。想要获取过去7天的文章？你无法直接筛选，除非每次查询时都先进行类型转换。想要按时间顺序排序并确保顺序正确？文本排序和日期排序不是一回事，根据格式的不同，它们可能会产生完全不同的结果。
Then there was the second problem, the one I almost missed entirely because it was hiding in plain sight. Look at those titles again: 接着是第二个问题，一个因为隐藏在眼皮底下而差点被我完全忽略的问题。再看看那些标题：
- Django Weblog: Last Call 2026 Django Developer Survey
- Mike Driscoll: New Book Release: Python Typing
Every single title in this feed follows the same pattern. Author or blog name, a colon, then the actual headline. That’s real, structured information sitting inside a single text field, completely unusable as a filter or a group-by. I couldn’t answer a question as simple as “which blogs post the most on Planet Python” because that information wasn’t a column. It was just text, buried. 这个订阅源中的每一个标题都遵循相同的模式：作者或博客名称，一个冒号，然后是实际的标题。这是真正的结构化信息，却被困在一个单一的文本字段中，完全无法作为筛选或分组的依据。我甚至无法回答“哪些博客在 Planet Python 上发布的内容最多”这样简单的问题，因为该信息不是一个独立的列，它只是被埋没在文本中。
So that was the actual state of things. Two pipelines built, data flowing in on schedule, and I still couldn’t answer basic questions about my own data. Loading it was never the finish line. I just hadn’t gotten to the starting point yet. 这就是现状。两个流水线建好了，数据按计划流入，但我仍然无法回答关于我自身数据的基本问题。加载数据从来都不是终点，我只是还没到达起点而已。
Why dbt, Specifically
为什么要用 dbt？
My first instinct was to just fix this in Python, since that’s the tool I already trust. Write a script that reads from articles, parses the dates, splits the titles, writes the results into new columns or a new table. And that would have worked, technically. But the more I thought about it, the more that felt like patching the same hole I’d already dug twice. Both of my pipelines were extract and load, full stop, and if I bolted transformation logic onto a Python script again, I’d just be adding a third untested, undocumented step to a system that already had two. 我的第一直觉是用 Python 来解决这个问题，因为这是我最信任的工具。写一个脚本读取文章，解析日期，拆分标题，并将结果写入新列或新表中。从技术上讲，这确实可行。但我越想越觉得，这就像是在我挖的同一个坑上打补丁。我的两个流水线都只有提取和加载，如果我再次将转换逻辑硬塞进 Python 脚本中，我只是在已经有两个步骤的系统中，又增加了一个未经测试、没有文档的第三步。
dbt does this differently, and that difference is kind of the whole point of the tool. Instead of a script that runs once and produces some output you have to trust blindly, dbt models are SQL that gets version controlled, tested, and documented as part of the same workflow. You write a transformation, and in the same project you can assert things about it: this column should never be null, this ID should always be unique. If those assumptions break, you find out immediately, not three weeks later when a chart looks wrong and you have no idea why. dbt 的做法则不同，而这种差异正是该工具的核心所在。dbt 模型不是那种运行一次、产生你必须盲目信任的输出的脚本，它是 SQL，可以作为同一工作流的一部分进行版本控制、测试和记录。你编写一个转换逻辑，并在同一个项目中对其进行断言：这一列绝不能为 null，这个 ID 必须唯一。如果这些假设被破坏，你会立即发现，而不是等到三周后图表出错却不知原因时才察觉。
It also matches how the industry actually works. Every data engineering job post I’ve looked at over the past two months mentions dbt, or something dbt-shaped. Learning it wasn’t just about fixing my RSS data, it was about learning the tool that’s become the default way teams handle the “T” in ETL. So instead of another Python script, I decided to actually sit down and learn dbt properly, on data I already had, with problems I already understood. Here’s how that went. 这也符合行业实际运作的方式。过去两个月我查看的每一个数据工程职位描述都提到了 dbt，或者类似 dbt 的工具。学习它不仅是为了修复我的 RSS 数据，更是为了掌握那个已成为团队处理 ETL 中“T”（转换）环节默认方式的工具。因此，我决定不再写另一个 Python 脚本，而是真正坐下来，用我现有的数据和已经理解的问题，好好学习 dbt。以下是我的学习过程。
Setting Up (and Immediately Hitting a Wall)
环境配置（并立即撞上南墙）
Getting dbt installed should have been the boring part. It wasn’t. I tried pip install dbt-postgres and got a wall of dependency resolution errors, dbt-core had no matching distribution for my environment. Turns out I was running Python 3.14, which is new enough that dbt hadn’t caught up to it yet. dbt Core officially supports up to 3.13 right now, and there’s usually a lag before it supports whatever Python just released. The fix wasn’t complicated once I understood the actual problem.
安装 dbt 本应是枯燥乏味的部分，但事实并非如此。我尝试运行 pip install dbt-postgres，结果遇到了一堆依赖解析错误，dbt-core 没有匹配我当前环境的发行版。原来我正在使用 Python 3.14，这个版本太新了，dbt 还没跟上。目前 dbt Core 官方最高支持到 3.13，通常在新版 Python 发布后，dbt 对其支持会有一定的滞后。一旦我理解了真正的问题所在，修复起来并不复杂。