X open sources its ranking algorithm, letting users see if they’ve been ‘shadowbanned’
X open sources its ranking algorithm, letting users see if they’ve been ‘shadowbanned’
X 开源其排名算法,允许用户查看自己是否被“影子禁言”
X is significantly expanding its open source codebase, which includes the app’s “For You” algorithm and its core ranking engine, and adding a feature that will let users see if their account or posts have been impacted by any of its ranking systems, the social network said on Thursday. X 公司周四表示,正在大幅扩展其开源代码库,其中包括该应用的“为你推荐”(For You)算法及其核心排名引擎,并增加了一项功能,让用户能够查看其账户或帖子是否受到了任何排名系统的影响。
The company is making the source code for the “For You” timeline, the default feed you see when you open the app, available on GitHub under the Apache v2 license. It’s also expanding its previous efforts to open source parts of its codebase to add more detail, including the model configuration, filter, and core ranking system details. That means it includes the parameters used to weight different signals — key to understanding which posts are actually displayed. This also makes the codebase roughly 10 to 15 times larger than it was before. 该公司正在以 Apache v2 许可证在 GitHub 上发布“为你推荐”时间线的源代码,即你打开应用时看到的默认信息流。它还扩展了之前开源部分代码库的努力,增加了更多细节,包括模型配置、过滤器和核心排名系统的详细信息。这意味着它包含了用于衡量不同信号权重的参数——这是理解哪些帖子被实际展示的关键。这也使得代码库的规模比以前扩大了大约 10 到 15 倍。
“You’ll get the core ranking code that pulls posts and ranks them for any given user and assembles the feed,” X’s VP of Product Keith Coleman told TechCrunch in an interview ahead of the announcement. “You can see the systems that filter out potentially problematic, rule-violating content…And some of those systems, like the ranker and the score, you can even run yourself outside the company.” “This is the kind of thing that I think people will be fairly shocked that we are releasing,” he added. “你将获得核心排名代码,它能为任何特定用户提取帖子、进行排名并组装信息流,”X 产品副总裁 Keith Coleman 在公告发布前接受 TechCrunch 采访时表示。“你可以看到过滤潜在问题和违规内容的系统……其中一些系统,比如排名器和评分系统,你甚至可以在公司外部自行运行。”他补充道:“我认为人们会对我们发布这些内容感到相当震惊。”
In addition to the repository, X is providing tools that will let users see for themselves if and how X’s ranking systems have impacted their account or posts. A new transparency tool is rolling out to an “Under the Hood” page in the app’s settings, which will let users who have posted 10 or more times over the past month download their aggregate stats as a JSON file. The file will show if any labels have been applied to their account or posts over the past calendar month. Non-technical users can take advantage of this information by dropping it into an LLM of their choice, pointing the AI at X’s GitHub repo, and asking for an interpretation. The company notes this tool will initially be available to a test group of accounts at least a year old as a pilot, before rolling out more broadly. 除了代码库,X 还提供了一些工具,让用户能够亲自查看 X 的排名系统是否以及如何影响了他们的账户或帖子。一项新的透明度工具正在推送到应用设置中的“幕后”(Under the Hood)页面,该工具将允许过去一个月发布过 10 次或以上帖子的用户下载其汇总统计数据的 JSON 文件。该文件将显示过去一个月内是否有任何标签被应用于他们的账户或帖子。非技术用户可以利用这些信息,将其输入到他们选择的大语言模型(LLM)中,指向 X 的 GitHub 代码库,并要求 AI 进行解读。公司指出,该工具最初将作为试点提供给一组注册时间至少一年的测试账户,然后再进行更广泛的推广。
Ahead of launch, the company previewed its open source codebase to external researchers familiar with recommendation systems, who were able to train and run X’s Phoenix scoring system using the open source code. That was a major milestone for X’s transparency efforts, Coleman says. From the GitHub repository, developers will be able to submit updates, known as pull requests, which X engineers will consider incorporating into its algorithm. While not all additions will make the cut, Coleman is enthusiastic about the idea. “That would be amazing to have people submitting code that improves the algorithm…I mean, how cool would it be for the X algorithm to be not just visible to the public, but also, like, by the public?,” he said. 在发布之前,该公司向熟悉推荐系统的外部研究人员预览了其开源代码库,他们能够使用这些开源代码训练并运行 X 的 Phoenix 评分系统。Coleman 表示,这是 X 透明度工作的一个重要里程碑。开发者将能够通过 GitHub 代码库提交更新(即 Pull Requests),X 的工程师将考虑将其整合到算法中。虽然并非所有提交都会被采纳,但 Coleman 对这个想法充满热情。“如果人们能提交改进算法的代码,那将是非常棒的……我的意思是,如果 X 算法不仅能被公众看到,还能由公众参与构建,那该有多酷?”他说。
However, a few systems are not included in this release, like those that use Grok to predict whether a post could be violating a rule. This is meant to protect X from bad actors who could use this information to work around the company’s rules to flood the network with spam. 然而,此次发布并未包含少数系统,例如使用 Grok 预测帖子是否可能违规的系统。此举旨在保护 X 免受恶意行为者的侵害,防止他们利用这些信息规避公司规则,从而在网络上发布垃圾信息。
The changes are meant to address continuing concerns about how X’s algorithm influences politics, elections, the spread of misinformation, and more. The platform has been home to political figures and high-profile individuals for years, and is now owned by a trillionaire who helped President Trump get elected. But Twitter had been under attack over its lack of transparency before it was bought by Musk, too. In earlier years, Republicans in Congress alleged that the California-headquartered social network leaned too far left, and claimed the network had “shadowbanned” their posts — meaning their posts were made invisible or undiscoverable without their knowledge. Twitter consistently denied this was the case. 这些变化旨在解决人们对 X 算法如何影响政治、选举、虚假信息传播等问题的持续担忧。多年来,该平台一直是政治人物和知名人士的聚集地,现在由一位曾帮助特朗普总统当选的亿万富翁拥有。但在被马斯克收购之前,Twitter 也曾因缺乏透明度而受到抨击。早些年,国会的共和党人曾指责这家总部位于加州的社交网络过于偏左,并声称该网络“影子禁言”了他们的帖子——即在他们不知情的情况下,使他们的帖子变得不可见或无法被发现。Twitter 一贯否认存在这种情况。
“Our dream is that anyone in the public can be able to assess how posts are distributed on the platform, vet that it’s a level playing field, and, if they think it’s not, critique it so we can keep improving it and addressing it,” Coleman said. “That’s the whole goal of this: [the code] can be audited; it can be critiqued. We’re going to listen, and we want to make the system one that people like and trust, and feel is fair,” he said. “我们的梦想是让公众中的任何人都能评估帖子在平台上的分发方式,验证这是否是一个公平的竞争环境;如果他们认为不是,可以提出批评,以便我们不断改进和解决问题,”Coleman 说。“这就是此举的全部目标:代码可以被审计,可以被批评。我们会倾听意见,我们希望让这个系统成为人们喜欢、信任并感到公平的系统,”他说。
Though X may be pushing itself to be more transparent around its code and other features, like its crowdsourced fact-checking system, Community Notes, it arguably became less transparent overall under Musk, after becoming a private company again. As it was no longer required to report to the SEC, X was not as forthcoming in other areas, like its user metrics, growth, revenue, or government takedown requests, which are now less frequently reported. Now merged with SpaceX, its monthly active users are once again public. 尽管 X 可能正在推动其代码和其他功能(如众包事实核查系统“社区笔记”)的透明度,但在马斯克将其私有化后,其整体透明度可以说有所下降。由于不再需要向美国证券交易委员会(SEC)报告,X 在其他领域(如用户指标、增长、收入或政府下架请求)的披露不再那么积极,这些数据的报告频率也降低了。目前,随着与 SpaceX 的合并,其月活跃用户数再次公开。
Correction: Included a clarification that X merged with SpaceX and now reports public numbers around user metrics. Coleman also said during the interview that external researchers were about to get “the whole score up and running themselves,” but the company clarified that they trained and ran the Phoenix scoring system themselves, but did not get the per-post score. 更正:补充说明 X 已与 SpaceX 合并,并开始公开报告用户指标数据。Coleman 在采访中还表示,外部研究人员即将能够“自行运行整个评分系统”,但公司澄清说,他们是自行训练并运行了 Phoenix 评分系统,但并未获得单条帖子的评分。