SkillTrace: Multi-Trace Provenance Auditing for LLM-Agent Skill Reuse
SkillTrace: Multi-Trace Provenance Auditing for LLM-Agent Skill Reuse
SkillTrace：面向大模型智能体技能复用的多轨迹溯源审计
Abstract: LLM-agent ecosystems are rapidly growing around reusable skills: mixed-modality packages of metadata, natural-language instructions, code, tools, references, and operational workflows. As skills become marketplace artifacts, auditing their reuse is no longer the same problem as ordinary code clone detection. Existing detectors target single-modality source code or whole-package similarity, yet skill reuse evidence is distributed across authored text, implementation fragments, and operational structure. As a result, they can miss reuse that preserves only one part of a skill.
摘要： 大模型智能体（LLM-agent）生态系统正围绕可复用技能迅速增长：这些技能是包含元数据、自然语言指令、代码、工具、参考资料和操作工作流的混合模态包。随着技能成为市场化的产物，审计其复用情况已不再等同于传统的代码克隆检测问题。现有的检测器主要针对单模态源代码或整个包的相似性，然而技能复用的证据分布在编写的文本、实现片段和操作结构中。因此，它们可能会遗漏那些仅保留了技能部分内容的复用行为。
We present SKILLTRACE, a multi-trace provenance auditing framework for LLM-agent skill reuse. SKILLTRACE extracts three provenance traces: Expression, Implementation, and Operational. It represents the Operational Trace as a Skill Operational Graph (SOG) that captures activation, procedure, and resource-flow structure. An LLM assists only the Operational-trace extraction, once at ingestion; at audit time SKILLTRACE compares cached traces deterministically, calibrates each trace against same-function strict negatives, and reports which trace supports a reuse decision.
我们提出了 SKILLTRACE，这是一个用于大模型智能体技能复用的多轨迹溯源审计框架。SKILLTRACE 提取了三种溯源轨迹：表达（Expression）、实现（Implementation）和操作（Operational）。它将操作轨迹表示为技能操作图（SOG），用以捕获激活、过程和资源流结构。大模型仅在摄入阶段辅助一次操作轨迹的提取；在审计时，SKILLTRACE 会确定性地比较缓存的轨迹，针对相同功能的严格负样本对每条轨迹进行校准，并报告是哪条轨迹支持了复用判定。
On SKILLTRACE-BENCH, with 820 transformed reuse positives over 100 marketplace anchors and 751 negative controls, SKILLTRACE achieves AUROC 0.938 and F1 0.898. A 36,446-skill wild audit further shows that trace-attributed evidence surfaces actionable reuse review queues beyond repository-level baselines.
在包含 100 个市场锚点、820 个转换后的复用正样本和 751 个负样本对照组的 SKILLTRACE-BENCH 基准测试中，SKILLTRACE 达到了 0.938 的 AUROC 和 0.898 的 F1 分数。一项针对 36,446 个技能的实地审计进一步表明，基于轨迹归因的证据能够提供比仓库级基准更具可操作性的复用审查队列。