How to Run a Chatbot on Your Own Computer

如何在自己的电脑上运行聊天机器人

It’s no exaggeration to say the large language models that power AI bots like ChatGPT and Gemini are changing the world. Many of us are now relying on them for coding, writing, summarizing, and searching the web. Whatever directions AI goes in over the next few years, LLMs will be part of that future. 毫不夸张地说，驱动 ChatGPT 和 Gemini 等人工智能机器人的大语言模型（LLM）正在改变世界。我们许多人现在都依赖它们进行编程、写作、总结和网络搜索。无论人工智能在未来几年走向何方，大语言模型都将是未来的一部分。

If you use ChatGPT, Claude, Perplexity, or any of the other AI platforms, then you’re using an LLM. What’s less commonly known is that these LLMs can be run locally, on your own computer. The key benefits are offline access and greater privacy, because you’re not sending anything to the cloud for anyone else to analyze or review. 如果你使用 ChatGPT、Claude、Perplexity 或任何其他人工智能平台，那么你就在使用大语言模型。鲜为人知的是，这些模型可以在本地，即你自己的电脑上运行。其主要优势在于离线访问和更高的隐私性，因为你无需将任何数据发送到云端供他人分析或审查。

You’re also not paying any AI companies a monthly subscription or hitting any usage rates. Numerous LLMs are available to download for free, including from big names like Meta and Google. Though they tend not to be as advanced or as speedy as the LLMs inside the apps you have to pay for, they’re capable enough for everyday use, and you can pick and choose between them as needed. 你也不必向任何人工智能公司支付月费，也不会受到使用频率的限制。许多大语言模型都可以免费下载，包括来自 Meta 和 Google 等大公司的模型。虽然它们通常不如你付费使用的应用程序中的模型那么先进或快速，但它们足以满足日常使用，你可以根据需要进行选择。

There is more maintenance involved here, and you lose some of the convenience that comes with just loading up the ChatGPT app. (You’ll need to handle updates yourself, for example.) However, you get a more personalized and private AI system all to yourself, and it’s not difficult to get started. 这需要更多的维护工作，而且你会失去直接打开 ChatGPT 应用程序所带来的部分便利（例如，你需要自己处理更新）。然而，你将拥有一个完全属于你自己的、更个性化且更私密的人工智能系统，而且上手并不困难。

What You Need to Get Started

入门所需准备

You can run local LLMs on Windows, macOS, and Linux, though macOS tends to be the preferred platform for most AI enthusiasts. Everything is more unified and consistent on Macs—unlike Windows PCs, only one company makes them—and Apple Silicon chips combine the CPU, GPU, and RAM together, which AI models like. 你可以在 Windows、macOS 和 Linux 上运行本地大语言模型，不过 macOS 通常是大多数人工智能爱好者的首选平台。Mac 上的软硬件更加统一和一致——与 Windows 电脑不同，Mac 只有一家公司生产——而且 Apple Silicon 芯片将 CPU、GPU 和内存整合在一起，这正是人工智能模型所青睐的架构。

Whichever platform you go for, having plenty of RAM helps. The bare minimum is 8 GB, though if that’s all you’ve got, you’ll be somewhat limited in terms of the size of the LLMs you can run and how fast they’ll go. Sixteen gigs is better, and 32 or more is required if you want to use the biggest and fastest models. For the best results, you need plenty of VRAM on a dedicated GPU (anything above 8 GB will make a difference), as this type of memory is optimized for the kind of tasks AI models do. 无论你选择哪个平台，充足的内存（RAM）都会有所帮助。最低要求是 8 GB，但如果只有这么多内存，你在运行大语言模型的大小和速度方面会受到一定限制。16 GB 会更好，如果你想使用最大、最快的模型，则需要 32 GB 或更多。为了获得最佳效果，你需要独立显卡（GPU）上有充足的显存（VRAM）（超过 8 GB 就会有明显差异），因为这种内存针对人工智能模型执行的任务进行了优化。

An AI chatbot like ChatGPT relies on the cloud. Photograph: David Nield 像 ChatGPT 这样的人工智能聊天机器人依赖于云端。摄影：David Nield

If you’re on Windows, a dedicated Nvidia GPU helps a lot: Graphics chips are better for running AI processes than standard processors, which is partly why Nvidia and the AI boom are so closely interlinked. These dedicated graphics cards will also have their own RAM, which gives AI models extra thinking space. 如果你使用的是 Windows，配备一块独立的 Nvidia 显卡会大有帮助：图形芯片在运行人工智能进程方面比标准处理器更出色，这也是 Nvidia 与人工智能热潮紧密相连的部分原因。这些独立显卡还拥有自己的显存，这为人工智能模型提供了额外的“思考空间”。

So while there’s no “minimum spec” for running a local LLM, as much RAM as possible and a discrete graphics card will certainly help. You then need a piece of software to run the model (to act as the outer interface), plus the model itself. There are plenty of choices for both, so you can pick and choose as required. 因此，虽然运行本地大语言模型没有严格的“最低配置”，但尽可能多的内存和一块独立显卡肯定会有所帮助。接下来，你需要一个运行模型的软件（作为外部界面），以及模型本身。这两者都有很多选择，你可以根据需要进行挑选。

Hugging Face offers more than 3 million models for download. Photograph: David Nield Hugging Face 提供超过 300 万个模型供下载。摄影：David Nield

For those getting started, LM Studio Bionic is generally considered to be the best AI app choice on Windows and macOS, and it’s free to use. Other popular and trusted options include vLLM, Llama.cpp, Ollama, and GPT4All, which are again available for multiple operating systems. These pieces of software are a little more technical and involved, though. 对于初学者来说，LM Studio Bionic 通常被认为是 Windows 和 macOS 上最好的人工智能应用程序选择，而且它是免费使用的。其他流行且值得信赖的选择包括 vLLM、Llama.cpp、Ollama 和 GPT4All，它们同样适用于多种操作系统。不过，这些软件的技术门槛稍高，涉及的操作也更多。

With an AI app chosen, the other decision is to settle on an LLM. The software programs we’ve mentioned will guide you toward some options, and there are also online repositories of models, the most well-known being Hugging Face (which has more than 3 million models available—quite the selection). 选好人工智能应用程序后，另一个决定就是确定要使用的大语言模型。我们提到的软件程序会引导你进行一些选择，此外还有在线模型库，其中最著名的是 Hugging Face（拥有超过 300 万个模型可供选择，种类非常丰富）。

An Example Local LLM Setup

本地大语言模型设置示例

A whole host of setups are possible when it comes to local LLMs, but we’ll take you step by step through the process of setting up LM Studio Bionic on Windows, to give you an idea of how it works. Once you’ve downloaded the installation file from the website and have LM Studio Bionic up and running, click Create Project to get started, and give your first project a name. 本地大语言模型有多种设置方式，但我们将带你逐步了解在 Windows 上设置 LM Studio Bionic 的过程，让你了解其工作原理。从网站下载安装文件并运行 LM Studio Bionic 后，点击“Create Project”（创建项目）开始，并为你的第一个项目命名。

A blank conversation window opens: If you click Choose a model in the prompt box, then Get local models, you’ll be taken to the main model picker for LM Studio Bionic. Each of the available models is listed together with its size, its popularity, and a few details about it—keep an eye out for the staff picks if you want some recommendations to get started with. Smaller models won’t be quite as capable, but they will download faster and take up less room on your system. 一个空白的对话窗口会打开：如果你点击提示框中的“Choose a model”（选择模型），然后点击“Get local models”（获取本地模型），你就会进入 LM Studio Bionic 的主模型选择器。每个可用模型都会列出其大小、受欢迎程度和一些详细信息——如果你需要入门建议，可以留意“staff picks”（员工精选）。较小的模型功能可能没那么强大，但下载速度更快，占用的系统空间也更少。

The model picker inside LM Studio. Photograph: David Nield LM Studio 内的模型选择器。摄影：David Nield

Back in the chat interface, everything works similar to a standard AI chatbot app. In the prompt box at the bottom you can choose between your AI models (if you have more than one LLM installed), and on the left of the prompt box you’ve got a + (plus) icon for submitting images and files (if this is supported by the current model). 回到聊天界面，一切的操作都与标准的人工智能聊天机器人应用程序类似。在底部的提示框中，你可以选择不同的人工智能模型（如果你安装了多个模型），在提示框左侧有一个“+”（加号）图标，用于提交图像和文件（如果当前模型支持的话）。

On the left of the interface is the navigation pane, where you can switch between your projects. Via the Settings link (lower left) on the navigation pane you’ve got the preferences screen for LM Studio Bionic. There are a bunch of ways to customize the software here, from how deleted chats are handled to the interface elements onscreen. 界面左侧是导航窗格，你可以在其中切换项目。通过导航窗格上的“Settings”（设置）链接（左下角），你可以进入 LM Studio Bionic 的偏好设置屏幕。这里有多种自定义软件的方法，从如何处理已删除的聊天记录到屏幕上的界面元素，应有尽有。

The LM Studio interface, running privately and offline. Photograph: David Nield LM Studio 界面，在私密和离线状态下运行。摄影：David Nield

Click Library in Settings to manage your existing models or the Explore menu entry to find new ones. If you need support for images and documents, look for multimodal models. (Technically, these build on top of the capabilities of an LLM, though they often get called LLMs anyway.) 点击设置中的“Library”（库）来管理你现有的模型，或点击“Explore”（探索）菜单项来查找新模型。如果你需要对图像和文档的支持，请寻找多模态模型。（从技术上讲，这些模型是在大语言模型的基础上构建的，尽管它们通常也被称为大语言模型。）

The final part of the LM Studio interface that is important to know about is the right-hand sidebar, which can be shown or hidden via the button on the very top-right corner. These give you further options for managing files across projects and giving the program access to the file system on your computer, if needed. LM Studio 界面中最后需要了解的重要部分是右侧边栏，可以通过右上角的按钮显示或隐藏。它为你提供了更多选项，用于管理跨项目的各种文件，并在需要时授予程序访问你电脑文件系统的权限。