Persuadability and LLMs as Legal Decision Tools

易受说服性与作为法律决策工具的大语言模型

Abstract: As Large Language Models (LLMs) are proposed as legal decision assistants, and even first-instance decision-makers, across a range of judicial and administrative contexts, it becomes essential to explore how they answer legal questions, and in particular the factors that lead them to decide difficult questions in one way or another.

摘要： 随着大语言模型（LLMs）被提议在各种司法和行政环境中担任法律决策助手，甚至是一审决策者，探索它们如何回答法律问题，特别是导致它们以某种方式决定难题的因素，变得至关重要。

A specific feature of legal decisions is the need to respond to arguments advanced by contending parties. A legal decision-maker must be able to engage with, and respond to, including through being potentially persuaded by, arguments advanced by the parties. Conversely, they should not be unduly persuadable, influenced by a particularly compelling advocate to decide cases based on the skills of the advocates, rather than the merits of the case.

法律决策的一个显著特征是需要对争议双方提出的论点作出回应。法律决策者必须能够参与并回应各方提出的论点，包括在必要时被其说服。反之，决策者不应过度易受说服，即不应受到某位特别有说服力的辩护人的影响，从而基于辩护人的技巧而非案件本身的事实依据来裁决案件。

We explore how frontier open- and closed-weights LLMs respond to legal arguments, reporting original experimental results examining how the quality of the advocate making those arguments affects the likelihood that a model will agree with a particular legal point of view, and exploring the factors driving these results. Our results have implications for the feasibility of adopting LLMs across legal and administrative settings.

我们探讨了前沿的开源和闭源权重 LLMs 如何回应法律论点，报告了原创的实验结果，考察了提出这些论点的辩护人的质量如何影响模型认同特定法律观点的可能性，并探索了驱动这些结果的因素。我们的研究结果对于在法律和行政环境中采用 LLMs 的可行性具有重要意义。