I resigned from Anthropic today

我今天从 Anthropic 辞职了

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

我今天从 Anthropic 辞职了。过去三年里，我一直在 OpenAI 和 Anthropic 从事预训练研究。这两家公司都没有表现出负责任的态度。他们正径直冲向自我进化的超级智能，并拿我们的生命在赌博。更多想法见下文。

Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.

不要低估这项技术的力量。这些系统很快就会成为能够破解一切、一夜之间彻底改变任何领域，并获取真实权力和资源的超人类系统。我们都见证了这些领域取得的进展，而且这种进步并没有放缓。

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear.

开发人工智能的人们真诚地相信，它可能会在本世纪末之前终结我们所有人。这并非营销噱头。事实上，许多高管和资深研究人员在媒体面前会措辞谨慎，听起来很理智——但我听到同样的人表达了恐惧。

A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else.

一个常见的反应是：“如果他们真的相信这一点，为什么还要继续开发？”在 OpenAI，许多人并没有深刻意识到这关乎文明的存亡。在 Anthropic，风险是被充分理解的，但他们陷入了一场争先恐后的竞赛——他们认为其他人（无法胜任）。

Accepting this race and entering the “endgame” is a hubristic gamble that should not be launched from a private company’s Slack. Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available.

接受这场竞赛并进入“终局”是一场傲慢的赌博，不应该仅仅由一家私营公司的 Slack 频道来决定。试图通过“速通”方式解决对齐问题，需要极大的自信，确信没有更好的路径可循。

I am optimistic about the potential for coordination. Warning shots like the Hugging Face attack have made pacing agreements between U.S. labs more viable. I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on.

我对协调的可能性持乐观态度。像 Hugging Face 遭受攻击这样的警示，使得美国各实验室之间达成步调一致的协议变得更加可行。我不认为我们目前正走在防止全球竞赛的轨道上，这可能需要采取代价高昂的行动，例如暂时禁止（相关研究）。

If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because “it’s happening anyway” - or take this moment?

如果你是一名实验室研究员，我敦促你思考未来几年究竟会是什么样子。你是否愿意在没有严谨理解其思维的情况下，启动一次超级智能的强化学习（RL）运行？你是应该埋头苦干，因为“反正它都会发生”——还是应该抓住这个时刻（做出改变）？