ShareLab-SII/VA-Judger

TL;DR: VA-Judger is a Qwen3-Omni-30B-based reward model that judges joint video-audio generations against human preferences — scoring not just per-modality quality but cross-modal semantic and temporal coherence — plus a full GRPO recipe for post-training LTX-2 with those rewards. The authors call it the first general reward model for joint AV generation, and the release is unusually complete: checkpoints, benchmark, inference code, SFT code, and an RL-tuned LTX-2 LoRA are all out; the VAPref-10K preference dataset and dimension-wise GRPO code are still pending.

Why this matters

RLHF for video generation is now standard, but joint video+audio generation has been stuck with cobbled-together reward signals: an audio quality metric here, a visual fidelity metric there, a sync score (AV-align style) on top. The paper's core claim is that this decomposition invites reward hacking — models learn to satisfy the individual metrics while producing content that's incoherent to humans, because no metric captures the joint prompt–video–audio relationship (does the moo happen when the cow strums?). Replacing the metric bag with a single omni-modal LLM judge trained on human pairwise preferences is the obvious right move, and this is the first serious open release doing it. If it holds up, it's the kind of infrastructure everyone building AV generation models will want.

How it works

The judge is Qwen3-Omni-30B-A3B fine-tuned to take a prompt plus two candidate AV clips and produce a chain-of-thought comparison, a preferred candidate, and 1–10 scores on five dimensions: (A) prompt matching, (B) audio-video consistency, (C) audio quality, (D) video quality, (E) content completeness/coherence.

Training is a three-stage curriculum: SFT on pairs with clear quality gaps to establish structured output and coarse discrimination; rejection-sampled CoT distillation on near-quality pairs, keeping only explanations whose verdicts match human annotations; then dimension-wise RL that decomposes the single binary preference label into per-dimension rewards for a denser signal. Data comes from VAPref-10K: 9K prompts, 10.3K paired comparisons across open-source generators.

For generator post-training, the clever bit is reward routing. LTX-2 has shared, audio, and video branches; VA-Judger's dimension scores map onto them. Per prompt, 8 candidates are generated and all 28 unordered pairs are judged; scores are normalized and routed:

What's actually there

Released: VA-Judger checkpoint (13 safetensors shards), RL-post-trained LTX-2 LoRA, VA-Judger-Bench (easy / in-domain / out-of-domain splits), reward-model SFT code (MS-Swift), inference for both models, and 8-GPU and 32-GPU GRPO launchers. Not yet: VAPref-10K and the dimension-wise GRPO code for the reward model itself — notably, the last training stage of their own method.

Evidence in the README is qualitative only: side-by-side demos (LTX-2 vs. OmniNFT vs. RL-tuned LTX-2) on the project page, and an abstract claiming VA-Judger beats metric baselines on human-preference prediction in- and out-of-domain. No numbers appear in the README — you'll need the paper (arXiv:2608.18607) for accuracy figures. The triage note is right that the key open question is whether the judge generalizes beyond the LTX-2/OmniNFT distribution it was trained against; the out-of-domain bench split is their answer, but it's self-reported.

What to expect

This is a serious-hardware release. The judge is a 30B MoE served via vLLM (one dedicated GPU minimum for reward serving); the default RL split is 1 reward GPU + 7 LTX-2 GPUs, scaling to 4 nodes × 8 GPUs. Reward-model SFT defaults to 8-GPU full-parameter with ZeRO-3 CPU offload (heavy host RAM). Three separate conda environments are needed because MS-Swift/vLLM versions conflict with the LTX-2 stack — the docs handle this honestly with VALIDATE_ONLY dry-run flags throughout, which is a nice touch. Gated HF access to upstream models (Qwen3-Omni, LTX-2, Gemma text encoder) is required. No license is stated in the README; the repo is ~1 week old, 57 stars, and the citation is a 2026 arXiv preprint.

The "first reward model for joint AV generation" claim is hedged ("to our knowledge") and plausible; OmniNFT (acknowledged as an influence) did modality-wise AV RL but with metric-based rewards, which is precisely what this replaces.

Try it

Shortest path is reward-model inference on the benchmark:

git clone https://github.com/ShareLab-SII/VA-Judger && cd VA-Judger/rewardmodel
conda env create -n va_reward_eval -f environment-eval.yml
conda activate va_reward_eval   # Python 3.12, MS-Swift 4.4.2, vLLM 0.19.0
hf auth login
bash scripts/evaluate.sh        # point BENCH_ROOT at easy/ indomain/ outdomain/ parent

Demos with audio are on the project page — worth two minutes to see whether the "Ours" column's moo timing convinces you before committing GPUs.