The Second Brain They Can’t Subpoena: Local RAG on a Pi 5

无法被传唤的“第二大脑”：在树莓派 5 上运行本地 RAG

If your memory is hosted, your thoughts are leased. We did not just move our files to the cloud. We moved our working memory. Andy Clark and David Chalmers called it the extended mind in 1998. The thesis was simple. Cognition leaks into the tools we trust. A notebook can be part of your mind if you access it reliably. 如果你的记忆被托管，那么你的思想就是租来的。我们不仅仅是将文件迁移到了云端，我们还迁移了工作记忆。Andy Clark 和 David Chalmers 在 1998 年提出了“延展思维”（extended mind）的概念。其论点很简单：认知会渗透到我们信任的工具中。如果你能可靠地访问一本笔记本，它就可以成为你思维的一部分。

In 2026, that notebook is a vector database owned by a platform with a legal department. Your extended mind now has terms of service, retention policies, and a compliance team that answers subpoenas faster than you answer email. I am not interested in nostalgia for paper. I am interested in architecture that preserves agency. The fix is not to think less with machines. It is to think locally with machines you control. 到了 2026 年，那本笔记本变成了一个由拥有法务部门的平台所控制的向量数据库。你的延展思维现在有了服务条款、保留策略，以及一个回复传票比你回复邮件还快的合规团队。我并不怀念纸质时代，我感兴趣的是能够维护自主权的架构。解决之道不是减少对机器的依赖，而是通过你所掌控的机器进行本地化思考。

That is why I built a second brain that lives on a Raspberry Pi 5 with NVMe and a Hailo-8 accelerator, running Retrieval Augmented Generation (RAG) completely offline. No API keys. No telemetry. No third party that can be compelled to hand over your associative graph. This is the expanded blueprint. More cohesive, more rigorous, and more useful than the usual cloud versus local sermon. 这就是为什么我构建了一个“第二大脑”，它运行在配备 NVMe 和 Hailo-8 加速器的树莓派 5 上，完全离线运行检索增强生成（RAG）。没有 API 密钥，没有遥测数据，也没有任何第三方可以被强制要求交出你的关联图谱。这是一份扩展蓝图，比常见的“云端 vs 本地”说教更具凝聚力、更严谨，也更有用。

The extended mind, now with a landlord

延展思维，现在有了“房东”

The original extended mind argument was about trust and coupling. If you reach for a tool as automatically as you reach for a memory, it counts as cognition. The cloud broke that coupling by inserting a landlord. Your retrieval is fast, but it is also observed, logged, ranked, and retained. 最初的“延展思维”论点是关于信任与耦合的。如果你像调用记忆一样自然地使用某个工具，它就构成了认知。云端通过插入一个“房东”打破了这种耦合。你的检索虽然迅速，但同时也处于被观察、记录、排名和保留的状态。

Three consequences follow. First, epistemic pollution. When your queries train their models, your future answers are shaped by everyone else’s queries. Your private context gets diluted by the median user. Second, legal exposure. Your prompts, your uploads, your retrieval history, and your embeddings are business records. In many jurisdictions they are discoverable. You cannot plead the fifth for data you gave to a provider. Third, strategic fragility. A policy change, a price hike, a region block, and your cognitive prosthesis goes dark. That is not a tool. That is a dependency. 这带来了三个后果。第一，认知污染。当你的查询被用于训练模型时，你未来的答案会被其他人的查询所塑造，你的私人语境会被普通用户的数据所稀释。第二，法律风险。你的提示词、上传内容、检索历史和向量嵌入都属于商业记录。在许多司法管辖区，这些数据是可以被调取的。对于你交给服务商的数据，你无法援引第五修正案（拒绝自证其罪）。第三，战略脆弱性。政策变更、价格上涨或区域封锁，都会导致你的认知假体瞬间瘫痪。那不是工具，那是依赖。

Local RAG restores the coupling. The model is on your desk. The index is on your disk. The retrieval path never leaves your LAN. You regain what philosophers care about and hackers need: direct, reliable, private access to your own prior thought. 本地 RAG 恢复了这种耦合。模型在你的桌面上，索引在你的硬盘里，检索路径永远不会离开你的局域网。你重新获得了哲学家所关心的、黑客所需要的核心：对你过往思想的直接、可靠且私密的访问权。

Why RAG beats fine tuning for a personal brain

为什么对于个人大脑而言，RAG 优于微调

Fine tuning bakes knowledge into weights. It is expensive, brittle, and hard to audit. RAG keeps knowledge outside the model and retrieves it at query time. For personal memory, this is superior for four reasons that matter intellectually, not just practically. 微调将知识固化在权重中，既昂贵又脆弱，且难以审计。RAG 将知识保留在模型之外，并在查询时进行检索。对于个人记忆而言，这在智识层面（而不仅仅是实用层面）有四个显著优势：

Provenance (溯源): RAG can cite the exact chunk it used. You can open the source note and verify. Fine tuned models hallucinate with confidence and no footnotes. 溯源：RAG 可以引用它所使用的确切片段。你可以打开源笔记进行核实。而微调模型往往会自信地产生幻觉，且没有脚注。
Mutability (可变性): Your life changes daily. With RAG you re-embed a note and the answer updates. With fine tuning you retrain or you live with stale weights. 可变性：你的生活每天都在变化。使用 RAG，你只需重新嵌入笔记，答案就会更新；而微调则需要重新训练，否则只能忍受陈旧的权重。
Composability (可组合性): You can mix corpora with metadata filters. Show me only work notes from 2024. Show me only code, not journals. This is information theory in practice. Retrieval is selective decompression. 可组合性：你可以通过元数据过滤器混合语料库。例如“只显示 2024 年的工作笔记”或“只显示代码而非日记”。这是信息论的实践，检索即选择性解压。
Portability (可移植性): A 2GB vector store and a quantized 8B model fit on a Pi. A personal fine tune that does not suck does not. 可移植性：一个 2GB 的向量存储库和一个量化的 8B 模型可以完美运行在树莓派上，而一个高质量的个人微调模型则做不到。

RAG is not a hack. It is a return to the original idea of hypertext, with similarity search instead of manual links. Bush’s Memex imagined associative trails. We finally have the math to build them. RAG 不是一种黑客手段，它是对超文本原始理念的回归，只不过是用相似性搜索取代了手动链接。Bush 的 Memex 曾构想过关联路径，我们终于拥有了构建它们的数学工具。

The architecture of uncompelled thought

不受胁迫的思想架构

Think in layers, not products. 按层级思考，而非按产品思考。

Ingest layer (摄入层): Files in, clean text out. PDFs via local OCR, web clips via readability, code via tree-sitter aware chunking. Every chunk gets metadata: source path, hash, created time, tags, and a privacy label. 摄入层：输入文件，输出纯文本。通过本地 OCR 处理 PDF，通过 readability 处理网页剪辑，通过 tree-sitter 感知分块处理代码。每个片段都带有元数据：源路径、哈希值、创建时间、标签和隐私标签。
Embedding layer (嵌入层): A small, local embedding model turns text into vectors. I use nomic-embed-text-v1.5 because it is compact, strong on recall, and runs fine on ARM. This is where most cloud setups leak. Do not leak here. 嵌入层：使用小型本地嵌入模型将文本转化为向量。我使用 nomic-embed-text-v1.5，因为它体积小、召回率高，且在 ARM 架构上运行良好。这是大多数云端方案泄露隐私的地方，千万不要在这里泄露。
Store layer (存储层): Qdrant on the Pi. It is written in Rust, low memory, and has good filtering. You want metadata filtering more than you want raw speed. Fast wrong answers are worse than slow right ones. 存储层：在树莓派上运行 Qdrant。它由 Rust 编写，内存占用低，且具备良好的过滤功能。相比原始速度，元数据过滤更为重要。快速的错误答案远不如缓慢的正确答案。
Model layer (模型层): Ollama serving a quantized instruct model. llama3.1:8b-instruct-q4_K_M is the sweet spot for a Pi 5 with 8GB. If you add Hailo-8, you can offload embedding inference and free CPU for generation. 模型层：使用 Ollama 运行量化指令模型。对于 8GB 内存的树莓派 5 来说，llama3.1:8b-instruct-q4_K_M 是最佳选择。如果添加 Hailo-8，你可以将嵌入推理卸载到加速器上，从而释放 CPU 用于生成任务。
Interface layer (接口层): A minimal FastAPI server that does retrieval, builds the prompt with citations, calls Ollama, and returns structured JSON. Your front end can be anything. I use a local Obsidian plugin and a TUI for field work. The entire loop stays on device. 接口层：一个极简的 FastAPI 服务器，负责检索、构建带引用的提示词、调用 Ollama 并返回结构化的 JSON。前端可以是任何东西，我使用本地 Obsidian 插件和 TUI（终端用户界面）进行外勤工作。整个循环都在设备上完成。

Hardware that makes this real

让这一切成为现实的硬件

The Pi 5 is not a toy anymore. The key change is PCIe exposed through the HAT connector. With a decent NVMe HAT you get real storage bandwidth, which is the actual bottleneck for RAG. 树莓派 5 已不再是玩具。关键的变化是通过 HAT 接口暴露出的 PCIe。配合一个不错的 NVMe HAT，你可以获得真正的存储带宽，这才是 RAG 的实际瓶颈所在。

Raspberry Pi 5, 8GB
Pineberry Pi HatDrive or similar NVMe HAT
1TB NVMe, TLC, DRAM cache preferred
Hailo-8 M.2 AI module, 26 TOPS at 2.5 watts
Aluminum passive case that doubles as heatsink
USB-C PD battery bank, 65W
Two microSD cards: one for bootloader, one for LUKS header backup

Power draw at idle is around 4 to 6 watts. Under generation it sits at 9 to 12 watts. You can run a full day on a 20,000 mAh bank. That is the point. A brain you cannot carry is a brain you will not use. 待机功耗约为 4 到 6 瓦，生成任务时约为 9 到 12 瓦。使用 20,000 mAh 的充电宝可以运行一整天。这就是重点：一个无法随身携带的大脑，你是不会去使用的。

Encrypt the NVMe with LUKS2. Use Argon2id, not PBKDF2. Store the keyfile on a USB drive you remove after boot, or memorize a strong passphrase. Mount the data partition noexec, nodev. Keep the OS on a read-only overlay so a hard power cut does not corrupt your root. This is not paranoia. It is systems hygiene. You are building a cognitive appliance, not a hobby box. 使用 LUKS2 加密 NVMe。使用 Argon2id 而非 PBKDF2。将密钥文件存储在启动后即可拔出的 USB 驱动器上，或者记住一个强密码。将数据分区挂载为 noexec 和 nodev。将操作系统保持在只读覆盖层上，以防断电导致根目录损坏。这不是偏执，这是系统卫生。你是在构建一个认知设备，而不是一个业余玩具。

A minimal, auditable software stack

极简且可审计的软件栈

Bloat is the enemy of auditability. Here is the compose file I run in production on the Pi. It is boring on purpose. 臃肿是可审计性的天敌。以下是我在树莓派上生产环境运行的 compose 文件。它故意设计得非常“无聊”（简单）。