I think we're cooked...

Honestly, tonight I had a realization moment. My token pan for qwen cloud expired, so I thought ‘you know what, I wanna see what local AI can do’. For context, I’m working on my IDE, so I took my buildbox (ryzen 5500, 32gb DDR4, GTX 1070 8gb, Ubuntu Server), using Drone, let my AI set up an Ollama instance, running Qwen 3.6 35b A3B Q5. 22gb total, sits comfortably in ram, A3B sits comfortably in 8gb vram with room to spare… 35 TPS… Sounds like nothing, but considering a word average is probably around 1.2 tokens… Suddenly that becomes a sentence a second.

老实说，今晚我突然意识到一件事。我的 Qwen 云端 Token 用完了，于是我想：“好吧，看看本地 AI 能做到什么程度。”为了说明背景，我当时正在处理我的 IDE，于是我用了我的开发机（Ryzen 5500, 32GB DDR4, GTX 1070 8GB, Ubuntu Server），通过 Drone 让我的 AI 设置了一个 Ollama 实例，运行 Qwen 3.6 35b A3B Q5 模型。总共 22GB，在内存中运行得很顺畅，A3B 模型在 8GB 显存中也绰绰有余……每秒 35 个 Token……听起来不多，但考虑到平均一个词大约是 1.2 个 Token……这突然就变成了一秒钟生成一个句子。

I imagine practically everyone reading this, given we’re developers and tend to put our money where it’s earned, into hardware, likely has a comparable system? 35 TPS, is about 1/5th what you can expect from a cloud model, except this is running on the hardware you already own and it doesnt stop you from using cloud too… The reason it made me question life as I know it, is because 35b params used to be the models of legend, the things that only ran in cloud, because nobody could afford a GPU large enough to run it… Now 8gb of vram is enough to get quite reasonable performance out of it… It changed how I saw open weight models and the future of the entire industry.

我想读到这里的每个人，鉴于我们都是开发者，倾向于把赚来的钱投入到硬件上，应该都有类似的系统吧？35 TPS 大约是云端模型性能的五分之一，但这是在你现有的硬件上运行的，而且这并不妨碍你同时使用云端服务……它让我质疑我所认知的现实的原因是，350 亿参数的模型曾经是传说级的存在，只有在云端才能运行，因为没人买得起那么大的 GPU……现在，8GB 显存就足以获得相当不错的性能了……这改变了我对开源权重模型以及整个行业未来的看法。

Go buy a 2nd hand RX 580 for $50, a scrap PC for $100, if ram wasnt so insanely expensive, $100 for 32gb of ram, now you added a new permanent node to your AI pool for $250 and it runs a damn good model, you can get away with half that if you go for a lighter model, or decide to offload execution to 1 system, instead of dispersing it to the nodes. Think about that, what you pay per year on AI subscriptions (or what some pay in 2 months), you can have a permanent AI added, no subscriptions and it doesnt stop at 1 node, add 10, add 100, it doesnt matter, the models are smart enough, that you’ll barely miss cloud models.

去买一张 50 美元的二手 RX 580，一台 100 美元的废旧电脑，如果内存没那么贵的话（32GB 内存 100 美元），你现在就以 250 美元的价格为你的 AI 集群增加了一个永久节点，而且它能运行非常棒的模型。如果你选择更轻量的模型，或者决定将执行任务集中在一个系统上而不是分散到各个节点，成本甚至可以减半。想想看，你每年在 AI 订阅上花的钱（或者有些人两个月花的钱），现在可以换成一个永久的 AI 节点，无需订阅，而且不仅限于一个节点，加 10 个、100 个都行，模型已经足够聪明，让你几乎感觉不到云端模型的缺失。

I just read an article too on how Halo got ported with AI acceleration and improved, Adobe and Microsoft’s suites are being redone in rust ALL AT ONCE! We’re reaching the point where code isnt cheap anymore, it’s worthlessly inexpensive, because for every name-brand, there’s a vibe-coded knockoff that likely performs better… And that’s scary as a developer, where are we all gunna be in 5 years? ‘But enterprise companies pay for software’, not if the knockoff is open source and they have AI too… To say we’re screwed is an understatement.

我刚读到一篇文章，讲的是《光环》（Halo）如何通过 AI 加速进行移植并得到改进，Adobe 和微软的套件正在被全面重写为 Rust 版本！我们正处于这样一个临界点：代码不再是廉价的，而是变得一文不值，因为每一个大牌软件背后，都有一个可能表现更好的“直觉编码”仿制品……作为开发者，这很可怕，五年后我们会在哪里？“但企业会为软件付费”，如果仿制品是开源的，而且他们也有 AI，那就不一定了……说我们完蛋了都算轻的。

And maybe it’s not a bad thing. The Halo port meant that the creative arts of making games is nearing it’s final call, software companies are pushing open source, because trying to protect your code is impossible now, we’re hitting the point where I honestly cant see what value any software can hold that a knockoff produced over a weekend doesnt. ‘Infrastructure’, it takes AI 10 minutes to get the same setup on Cloudflare, it’s as secure, if not more than what you built. ‘Quality’, they set the agent up to patch it continuously. ‘reliability’, the rewrote it in rust, it takes less memory, runs faster and is memory safe by default. ‘but mine looks better’, they’ve used screenshots and replicated GUIs perfectly for years…

也许这并不是坏事。《光环》的移植意味着游戏创作的艺术正接近终点，软件公司正在推动开源，因为现在想要保护代码已经不可能了。我们正处于这样一个阶段：我实在看不出任何软件能有什么价值是周末产出的仿制品所不具备的。“基础设施”方面，AI 用 10 分钟就能在 Cloudflare 上完成同样的设置，而且和你自己构建的一样安全，甚至更安全。“质量”方面，他们设置了代理来持续修补。“可靠性”方面，他们用 Rust 重写了它，占用内存更少，运行更快，且默认内存安全。“但我的看起来更好看”，他们多年来一直使用截图并完美复制 GUI……

What can we honestly provide today, that AI cant do better, cheaper, faster? Even IT support is becoming a thing of the past, hell I built Drone exactly to do that, now I dont even troubleshoot myself anymore, I just let my AI do it for me, cuz it saves me the hassle. The part that should really scare you, is what comes after we’re replaced. If it can rewrite adobe’s suite, microsoft’s suite, build an OS in 12 hours and write a complete ERP system in less than that, how much effort do you really think it takes to automate it’s use?

今天我们到底能提供什么 AI 做不到更好、更便宜、更快的事情？甚至 IT 支持也正在成为过去式。见鬼，我构建 Drone 正是为了做这个，现在我甚至不再自己排查故障了，我直接让 AI 代劳，因为它省去了我的麻烦。真正应该让你感到害怕的是，在我们被取代之后会发生什么。如果它能重写 Adobe 套件、微软套件，在 12 小时内构建一个操作系统，并在更短时间内写出一个完整的 ERP 系统，你觉得自动化它的使用还需要多少努力？

‘But surely entertainment is safe?’, CGI replaced practical effects 20 years ago and AI generated content trends longer and harder than human created content today. Take Gangnam Style, it broke records… But that Ai generated trumpet sound used on TikTok… How many times do you think THAT’s been heard? 1 billion times? Or closer to 10 billion times? The global economy has already shifted and we’re all still playing catchup. The irony is, as anyone who works a desk job, where’s our future? Thank goodness it wont happen overnight and the water is barely coming to a boil, but if there’s any advice I can give anyone, it’s learn a physical skill for the day using a computer wont earn you a penny, because it might come sooner rather than later.

“但娱乐业总该是安全的吧？”20 年前 CGI 取代了实景特效，而今天 AI 生成的内容比人类创作的内容流行得更久、更猛。以《江南 Style》为例，它打破了记录……但 TikTok 上那个 AI 生成的小号声……你觉得那被听过多少次？10 亿次？还是接近 100 亿次？全球经济已经发生了转变，而我们所有人还在苦苦追赶。讽刺的是，作为任何从事文职工作的人，我们的未来在哪里？谢天谢地，这一切不会在一夜之间发生，水温才刚刚开始升高，但如果我能给任何人什么建议，那就是去学习一项体力技能，以应对使用电脑赚不到一分钱的那一天，因为那一天可能比预想的来得更快。

Sorry if all of this sounds super pessimistic, I imagine in a way, it’s actually optimistic, if not for the fact that it means we’re all on the chopping block… What are your thoughts?

如果这一切听起来非常悲观，我很抱歉。我想在某种程度上，这其实是乐观的，如果不是因为这意味着我们都已经在砧板上了……你们怎么看？