Yzmblog/DMAD

TL;DR: DMAD is a ByteDance/TAMU diffusion-distillation method that removes the most awkward piece of DMD-style distillation — the auxiliary "fake score" diffusion model that must be continually re-fit to the student — and replaces it with two discriminator heads whose logits directly supply the log-density ratios the distribution-matching gradient needs, with a proof that at discriminator optimum you recover exactly the DMD gradient. The headline artifact: 4-step LoRA students of MiniMax-H3 (33B) doing 1344×768 joint text-to-audio-video with stereo sound, with full training code, checkpoints, a bit-exact 24 GB consumer-GPU path, and ComfyUI/Diffusers integrations.

The idea

Distribution Matching Distillation (DMD/DMD2) is the dominant recipe for few-step students of big diffusion models, but it carries a tax: alongside the student you must train a second full-size diffusion model fitted to the student's evolving output distribution, because the matching gradient is a difference of scores (teacher score minus student score). At 33B parameters that auxiliary model is a serious memory and compute cost.

DMAD's observation is the classical GAN identity: an optimal discriminator's logit is a log-density ratio, and the gradient of a log-density ratio is a difference of scores. So instead of fitting an auxiliary score model, attach two classifier heads to a shared backbone — one separating real data from student samples, one separating teacher samples from student samples — and train the student with linear losses on their logits. The paper proves these losses recover DMD's distribution-matching gradient at the discriminator optimum. A second contribution, gap-based reweighting, uses the real-data head's empirical logit gap between real and teacher samples to decide, per noise level, how much to trust the teacher versus the data — a sensible knob, since teachers are themselves imperfect at some noise levels.

Evidence

Numbers from the abstract (self-reported, but across three very different scales): FID 1.04 one-step on ImageNet-64, 14.47 four-step SDXL on COCO-10K, VBench 85.15 with four-step Wan2.1-T2V-14B — claimed best among compared few-step methods and the multi-step teachers, which is the usual GAN-distillation story of adversarial pressure sharpening past the teacher. On MiniMax-H3-33B audio-video, human raters prefer DMAD's 4-step student at 79.1% over DMD2 and 84.6% over rCM (ties excluded).

What's in the repo

This is unusually complete for a distillation paper. Two rank-128 LoRAs (1.4 GB each) on all 50 H3 transformer blocks: lora_critic (the paper's checkpoint) and full_critic (fully-trained critic backbone, scores higher on AVGen-Bench). An inference.py implementing the re-noise step rule the students were trained with (the official pipeline's Euler rule works but scores lower). Diffusers and ComfyUI integrations. Training code for all four model families (ImageNet-64/SDXL on DMD2's codebase, Wan2.1 on rCM's, H3 on LightX2V).

The engineering in the low-VRAM path deserves a note: --low-vram streams the 32B text encoder and 33B transformer layer-by-layer through rotating buffers, pre-computes the step-dependent AdaLN projections, and produces bit-identical output to the resident path (md5-verified) at 12.9 GiB peak during sampling. On an H200 the streamed path matches the resident path at 37 s per video; they estimate 4–8 minutes of sampling on a 4090/3090. An optional int8 weight mode halves traffic — different samples (4-step students amplify any weight perturbation) but, they claim, same scene-level quality.

Because training is continuous-time, --steps is free: 8–12 steps render fast motion (boxing, dunks) cleaner than 4.

Caveats

Quality claims are self-reported and the repo is new (~95 stars); the "beats the multi-step teacher on VBench" claim merits your own eyes on outputs. You need the ~170 GB MiniMax-H3 download under its Community License; the text encoder alone wants 62 GiB on GPU unless you use the streaming paths. Root code is Apache-2.0, but train/image inherits DMD2's CC BY-NC-SA. The paper checkpoint is from iteration 800 of training — early-stopping EMA, which hints the adversarial training may be delicate.

Try it

conda create -n dmad_h3 python=3.12 -y && conda activate dmad_h3
pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 \
  --index-url https://download.pytorch.org/whl/cu126
pip install -r requirements.txt

hf download MiniMaxAI/MiniMax-H3 --local-dir models/MiniMax-H3 \
  --exclude "FL2VA/*" --exclude "Ref2VA/*" --exclude "transformer_ref/*"
hf download ZhengmingYu/DMAD --include "minimax_h3/*" --local-dir ckpt

python inference.py --model-dir models/MiniMax-H3 \
  --lora ckpt/minimax_h3/dmad_minimax_h3_4step_lora_critic.safetensors \
  --prompt "A polar bear is playing the violin in the snow." \
  --output-dir outputs/polar_bear --low-vram   # 24 GB GPU; drop flag on 80 GB

Checkpoints and ComfyUI conversions: huggingface.co/ZhengmingYu/DMAD.