m-a-p/YuE2-3B

TL;DR: YuE2-3B is the follow-up to YuE, an open lyrics+style→full-song model (48 kHz stereo, vocals + accompaniment) that the authors claim matches or beats Suno v5/v6 on automatic benchmarks. The interesting part isn't just the quality: it's the interface. The model first writes an editable ABC-notation score (melody + chords) as an explicit symbolic plan, then renders it — which turns song generation into something you can inspect, edit by hand, or revise via an LLM agent loop, and which makes zero-shot covers a matter of feeding in a transcribed melody. Runs locally in ~11 GB VRAM on a 24 GB GPU.

What's actually new here

Most open song generators (ACE-Step, DiffRhythm, LeVo) go straight from text to audio tokens. YuE2 inserts a symbolic planning stage — chain-of-thought, but the thoughts are a score. One AR–NAR Mixture-of-Transformers backbone autoregressively writes the ABC score and semantic tokens; flow matching then generates acoustic latents, and a VAE decodes to stereo audio. The plan is a first-class artifact: pipe.plan() returns it, save_artifacts() dumps the ABC alongside tokens/latents/audio, and you can supply your own ABC (abc=...) to condition generation.

The agentic editing demo is the best illustration: one song revised through 9 steps / 14 versions, from Mandarin pop to English jazz with a saxophone solo quoting "Twinkle, Twinkle" — the agent rewrites the ABC/lyrics/style, YuE2 re-renders, and the whole conversation is inspectable. Nobody else in open music gen has shipped this workflow.

Evidence

On WildSongBench (192 prompts, automatic metrics), YuE2 best-of-8 posts the highest SongBench average of everything they evaluated:

Read the protocol notes carefully before repeating "beats Suno": best-of-8 uses selection by Musicality→Q3O→PER, all metrics are automatic (no human eval reported), Suno v5 still leads on MuLan/AllMusicCaps text-alignment, and the benchmark numbers use YuE2-Vae-legacy while the recommended default VAE trades benchmark musicality for perceptual quality. Single-sample YuE2 (6.73) sits between Suno v6 and v5. Still — an open 3B model in that bracket is a real jump.

The cover result is arguably the stronger evidence for the symbolic-plan thesis. On SHS100K zero-shot covers (3,792 songs, no cherry-picking), score-conditioned YuE2 hits 0.647 CLEWS mAP / 71.3% Hit@1 for preserved song identity vs 0.419/48.4% for SongEcho — and the ablation is clean: drop the score and identity collapses to 0.006 mAP while style similarity goes up. The score is doing the work.

What's there and what to expect

• Weights (safetensors, CC BY-NC 4.0 — non-commercial, note for any product use) plus two VAEs and a SheetSage2 transcriber as sibling repos. No technical report yet; the card says cite YuE 1 for now.
• Inference ships as a pre-built wheel downloaded from the HF repo (yue2_infer-0.1.5-py3-none-any.whl) with custom_code — you're pip-installing a binary from a model repo, so the usual trust considerations apply, and the training code/data aren't released.
• Speed is genuinely good: 3.6-min song in ~71 s on an RTX 4090 at ~11 GB peak VRAM (BF16 AR/NAR, FP32 VAE, CUDA graphs + FlashAttention). A separate vLLM serving path does ~373 songs/hour on an H800 at 32-way concurrency.
• Covers require an external transcription pipeline (SheetSage2 + an ASR for lyrics) in separate environments — workable but not one-click.
• Languages: English and Mandarin.

Try it

Linux, Python 3.10+, 24 GB NVIDIA GPU with BF16:

pip install huggingface-hub==0.36.2
hf download m-a-p/YuE2-3B yue2_infer-0.1.5-py3-none-any.whl --local-dir .
pip install ./yue2_infer-0.1.5-py3-none-any.whl

from yue2 import YuE2Pipeline
pipe = YuE2Pipeline.from_pretrained("m-a-p/YuE2-3B", device="cuda")
song = pipe(style="Funk, nu-disco, female vocal", lyrics=lyrics, cot="full")
song.save("song.flac")
song.save_artifacts("outputs/song")  # includes the editable ABC score

Or just listen first: demos (including the full 14-version editing trail) are linked from the model card.