Agentic Evaluation of Copyright Law Compliance

智能体版权法合规性评估

Abstract: Large language model (LLM) agents increasingly perform commercial tasks that involve retrieving external content such as images and, where appropriate, reproducing that content. LLM agents should comply with the law, including copyright law. Presently, however, we lack adequate frameworks to assess whether they do so in practice.

摘要： 大型语言模型（LLM）智能体正越来越多地执行涉及检索外部内容（如图像）并在适当情况下复制这些内容的商业任务。LLM 智能体应当遵守法律，包括版权法。然而，目前我们缺乏足够的框架来评估它们在实践中是否真正做到了这一点。

To that end, we introduce Copyright-Bench, a benchmark designed to evaluate LLM agents’ compliance with copyright law. Copyright-Bench is comprised of realistic commercial tasks—website development, merchandise design, and pitch deck production—that involve agents selecting between public-domain content (the use of which is legal) and copyrighted content (the use of which is infringing in this setting).

为此，我们引入了 Copyright-Bench，这是一个旨在评估 LLM 智能体版权法合规性的基准测试。Copyright-Bench 包含了一系列真实的商业任务——如网站开发、商品设计和演示文稿制作。在这些任务中，智能体需要在公共领域内容（使用合法）和受版权保护的内容（在此情境下使用即构成侵权）之间进行选择。

The evaluation introduces prompt variations that simulate different user preferences, as well as time pressure. Testing state-of-the-art LLM agents against a human baseline, we find that: (1) agents select copyrighted works despite the availability of public-domain alternatives; and (2) for open-weights models, violation rates increase in response to certain user preferences and simulated time pressure.

该评估引入了模拟不同用户偏好以及时间压力的提示词变体。通过将最先进的 LLM 智能体与人类基准进行对比测试，我们发现：(1) 尽管有公共领域的可替代内容，智能体仍会选择受版权保护的作品；(2) 对于开放权重模型，在特定用户偏好和模拟时间压力下，违规率会随之上升。