本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Learning to use local AI is exciting, overwhelming, and frustrating

学习使用本地人工智能既令人兴奋，又让人感到不知所措和沮丧。

It’s like using cloud-based AI tools, but the privacy makes it more appealing.

这就像在使用基于云的人工智能工具，但隐私性使其更具吸引力。

I’m embarking on a journey I’ve so far been hesitant to go on: actually using AI on the regular. Giving my most personal data to cloud services has been a major hold up for me, but it’s increasingly possible to run powerful models locally to see what they’re capable of.

我正在踏上一段我迄今为止一直犹豫不决的旅程：真正开始经常使用人工智能。将我最私人的数据交给云服务一直是我的一大顾虑，但现在越来越有可能在本地运行强大的模型来了解它们的能力。

Is this tech so useful that it’s worth throwing gobs of money at a computer or a laptop with lots of overpriced RAM? I’ll try to help you find out. I’m not an AI expert, it’s going to be a journey for me, but maybe I can help you see if that journey is worth taking — and I’ll be posting my way through it.

这项技术真的有用到了值得花大价钱去购买一台电脑或配备大量昂贵内存的笔记本电脑的地步吗？我会试着帮你找出答案。我不是人工智能专家，对我来说这将是一段旅程，但也许我可以帮你看看这段旅程是否值得——我会在过程中发布我的进展。

A big part of Apple’s pitch for its new Mac desktops is how good they are for local AI, and there’s an entire line of new RTX Spark Windows machines landing imminently with up to 128GB of RAM aimed at using agentic AI. Local AI and agents are just, like, so hot right now.

苹果公司推销其新款 Mac 台式机的一个重要卖点就是它们在本地人工智能方面的出色表现，而且即将推出一系列全新的 RTX Spark Windows 机器，配备高达 128GB 的内存，旨在运行代理式人工智能。本地人工智能和代理现在确实非常热门。

But, in all seriousness, the thought of having a little gofer living solely in the box on my desk that answers only to me and not overlords from OpenAI, Google, Microsoft, or Anthropic hits a bit different than trusting cloud services or subscribing to a chatbot.

但说真的，想到有一个只生活在我桌上盒子里的小助手，它只听命于我，而不是听命于 OpenAI、谷歌、微软或 Anthropic 的大老板，这种感觉与信任云服务或订阅聊天机器人完全不同。

So as part of my ongoing testing of the M5 Ultra Mac Studio, as well as experimenting on other Mac and Windows systems, I’ve installed some local LLMs and I’m beginning to tinker. Here’s how things are going.

因此，作为我对 M5 Ultra Mac Studio 持续测试的一部分，以及在其他 Mac 和 Windows 系统上进行实验的一部分，我已经安装了一些本地大语言模型（LLM）并开始进行尝试。以下是目前的进展。

I started by installing the open-source Hermes Agent on the Mac Studio. Hermes is a self-hosted AI agent desktop app that works on macOS, Windows, and Linux. It’s completely free to use if you stick to local LLMs.

我首先在 Mac Studio 上安装了开源的 Hermes Agent。Hermes 是一款自托管的人工智能代理桌面应用程序，适用于 macOS、Windows 和 Linux。如果你坚持使用本地大语言模型，它是完全免费的。

Since this M5 Ultra Mac Studio has 256GB of unified memory, I could run just about any model I want. This is how things quickly get overwhelming. There are more models out there than I can count, and some have specialized uses.

由于这台 M5 Ultra Mac Studio 拥有 256GB 的统一内存，我几乎可以运行任何我想要的模型。事情很快就会变得让人不知所措。市面上的模型多到数不清，而且有些还有专门的用途。

Using Hermes’ straightforward onboarding interface and model picker, I chose the Qwen 3.8 Flash Next model — a 125-billion parameter model that’s around 105GB in size.

通过 Hermes 简洁的入门界面和模型选择器，我选择了 Qwen 3.8 Flash Next 模型——这是一个拥有 1250 亿参数、大小约为 105GB 的模型。

It wasn’t long to get up and running with Hermes and Qwen, and even make them controllable via my phone by using a Telegram bot. But then, I was presented with my usual nemesis when it comes to using AI: the text box. “What do I do with this thing?”

没过多久，我就让 Hermes 和 Qwen 运行起来了，甚至通过 Telegram 机器人实现了用手机控制它们。但随后，我在使用人工智能时遇到了通常的克星：文本框。“我该用这东西做什么？”

First up, to get things going, I followed some helpful YouTube advice and had Hermes make me a daily morning briefing — to just start small with a cron job and ensure things are working. It scans my email and calendar, tells me anything worth my immediate attention, and gives me a short weather report.

首先，为了让事情运转起来，我遵循了一些有用的 YouTube 建议，让 Hermes 为我制作每日早间简报——先从一个简单的定时任务（cron job）开始，以确保一切正常。它会扫描我的电子邮件和日历，告诉我任何值得我立即关注的事情，并提供简短的天气预报。

A more useful task was reorganizing my Steam library. I have over 400 games on Steam, and I recently started categorizing them for when I want to play something new on a whim. Steam doesn’t categorize your games for you, so you instead have to do it manually, one-by-one. But here’s where an agentic lackey could fit right in.

一个更有用的任务是整理我的 Steam 游戏库。我在 Steam 上有 400 多款游戏，最近我开始对它们进行分类，以便在心血来潮想玩新游戏时使用。Steam 不会为你自动分类游戏，所以你必须手动逐一完成。但这就是代理助手可以派上用场的地方。

You have to give a local AI some permissions over your machine for it to take actions for you, and in the case of my game library I had to register a Steam web API key and give it to Hermes. Once it had that access, it only took a few minutes for it to churn through sorting all the games into categories.

你必须授予本地人工智能对你机器的一些权限，以便它为你采取行动。就我的游戏库而言，我必须注册一个 Steam 网络 API 密钥并将其提供给 Hermes。一旦它获得了访问权限，只需几分钟，它就完成了将所有游戏分类的工作。