multimodal-art-projection / YuE

Looking for the original YuE? Its code, documentation, and license are preserved on the YuE-v1 branch. 正在寻找最初的 YuE？其代码、文档和许可证已保留在 YuE-v1 分支中。

YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality

YuE2：以前沿质量统一符号与音频音乐生成

Compose in symbols. Create in sound. 🎧 Demos · 🤗 YuE2 · 🚀 Quick start · 🤖 Agent skill · 📊 Benchmarks · 🤗 MERT2 · 🤗 SheetSage2 · 🤗 WSB · 📦 Release 以符号构思，以声音创作。🎧 演示 · 🤗 YuE2 · 🚀 快速开始 · 🤖 智能体技能 · 📊 基准测试 · 🤗 MERT2 · 🤗 SheetSage2 · 🤗 WSB · 📦 发布

YuE2 brings frontier song quality to music generation with an editable composition. Give it lyrics and a style prompt: it writes a melody-and-chord plan, then realizes that plan as a complete song with vocals and accompaniment. YuE2 通过可编辑的乐谱，将前沿的歌曲质量带入音乐生成领域。只需提供歌词和风格提示词：它会编写一份旋律与和弦方案，然后将该方案实现为包含人声和伴奏的完整歌曲。

Frontier quality. YuE2 is competitive with Suno v5/v6 on WildSongBench. YuE2 (best-of-8) achieves 6.9632 SongBench Avg, the highest observed mean among all evaluated settings. 前沿质量。在 WildSongBench 测试中，YuE2 的表现可与 Suno v5/v6 相媲美。YuE2（8选1最佳）达到了 6.9632 的 SongBench 平均分，是所有评估设置中观察到的最高平均值。

White-box music generation through symbolic planning. Read, play, and change the composition before rendering it. Melody and chords become explicit controls that a person or an agent can inspect and edit. 通过符号规划实现“白盒”音乐生成。在渲染之前，你可以阅读、播放并修改乐谱。旋律与和弦成为明确的控制参数，供人类或智能体进行检查和编辑。

Zero-shot covers and agentic editing. Reimagine a transcribed song in a new style, or refine a song through a conversation about its score, arrangement, and lyrics—all with the same generation checkpoint. 零样本翻唱与智能体编辑。你可以用新的风格重新演绎一首转录的歌曲，或者通过关于乐谱、编曲和歌词的对话来精修一首歌曲——所有这些都使用同一个生成检查点（checkpoint）即可完成。

How it works

工作原理

One AR–NAR Mixture-of-Transformers backbone predicts the score and semantic tokens autoregressively, then generates acoustic latents with flow matching. A VAE decodes those latents into stereo audio. 一个 AR-NAR（自回归-非自回归）混合 Transformer 主干网络以自回归方式预测乐谱和语义标记，随后通过流匹配（flow matching）生成声学潜变量。最后，VAE 将这些潜变量解码为立体声音频。

Creation, covering, and editing differ in where the score comes from: YuE2, a transcribed recording, or an edited composition. The staged Python API exposes plan() → generate_semantic() → synthesize() → decode(). 创作、翻唱和编辑的区别在于乐谱来源不同：分别来自 YuE2 生成、转录录音或人工编辑的乐谱。分阶段的 Python API 提供了 plan() → generate_semantic() → synthesize() → decode() 的调用流程。

Quick start

快速开始

Linux · Python 3.12 · NVIDIA GPU with BF16 support and 24 GB VRAM. YuE2 produces 48 kHz stereo audio without quantization. Model files download from Hugging Face on first use. Linux · Python 3.12 · 支持 BF16 的 NVIDIA GPU 及 24 GB 显存。YuE2 可生成 48 kHz 的立体声音频，无需量化。模型文件在首次使用时从 Hugging Face 下载。

git clone https://github.com/multimodal-art-projection/YuE.git
cd YuE
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install .
python examples/generate.py --output outputs/first-song

Open outputs/first-song/audio.flac. The output directory also retains the score, semantic tokens, acoustic latents, generation settings, and model identities. 打开 outputs/first-song/audio.flac。输出目录还会保留乐谱、语义标记、声学潜变量、生成设置以及模型标识信息。

Cover a song

翻唱歌曲

Transcribe a source recording with 🤗 SheetSage2, review its melody ABC, and provide new lyrics or a target style. For covers, use cot="melody" and a score without chord symbols so the accompaniment can adapt to the new style. 使用 🤗 SheetSage2 转录源录音，检查其旋律 ABC 乐谱，并提供新的歌词或目标风格。对于翻唱，请使用 cot="melody" 并提供不含和弦符号的乐谱，以便伴奏能够适应新风格。

Edit a composition

编辑作品

Export a plan, revise the musical details, and render the edited score. The editable score is the white-box interface: you can inspect the intended composition and intervene on it. Editing generates a new complete recording; it does not preserve the original waveform outside an edit. 导出方案，修改音乐细节，并渲染编辑后的乐谱。可编辑的乐谱即是“白盒”接口：你可以检查预期的作品并进行干预。编辑操作会生成一首全新的完整录音；它不会保留编辑之外的原始波形。

Agent skill

智能体技能

The yue2-music skill teaches an agent how to generate songs, transcribe and cover recordings, edit ABC scores, check musical invariants, and organize listening comparisons. yue2-music 技能教会智能体如何生成歌曲、转录和翻唱录音、编辑 ABC 乐谱、检查音乐不变性以及组织听感对比。