HVision-NKU/OraRL

TL;DR: OraRL is a training recipe plus released 4B/9B "Video-ORA" models (Qwen3.5 backbones) that scales GRPO-style RL across seven video task families — temporal/spatial grounding, segmentation, tracking, spatio-temporal grounding, video QA, spatial intelligence — with one update rule. The genuinely interesting bit is annotations-as-rollouts: the ground-truth annotation is serialized and injected into each rollout group as a guaranteed-positive sample, while the advantage baseline is estimated from on-policy samples only. No chain-of-thought supervision, no CoT decoding at inference, full veRL-based training stack released under Apache-2.0.

The idea

Standard GRPO for video MLLMs has a cold-start problem: on hard tasks (say, pixel-level video segmentation or long-video temporal grounding), the policy may never sample a high-reward rollout, so the group advantage collapses to noise and learning stalls. The usual fixes are SFT warm-up or distilled CoT traces — both expensive and off-policy in an uncontrolled way.

OraRL's move is to notice that the dataset already contains a perfect rollout for every prompt: the annotation itself. Timestamps, boxes, masks, and answers are task-native and can be serialized into the same output format the policy emits. So each group gets exactly one "oracle" rollout appended to the K policy samples. The subtlety that makes this a real method rather than a hack: the group baseline is computed only from policy rewards, keeping the advantage estimate on-policy. The annotation-vs-policy reward gap is then converted into a separate correction term, and a sign-balanced subset of samples is retained per update — which is also where the efficiency gains come from.

What's released and what the evidence looks like

The release is unusually complete: Video-ORA-4B and -9B weights on Hugging Face (vLLM 0.19.1 compatible, OpenAI-serving out of the box), full training recipes (orarl_4b.yaml, orarl_9b.yaml) on a modified veRL stack, evaluation harness with a smoke test and full paper suite, live demo, and an "OraRL-Data" evaluation dataset. The infra section is worth noting for anyone doing video RL: decode-once frame reuse, task-grouped batching, async Ray rewards, and a unified video contract flowing through vLLM rollouts and FSDP updates — plumbing that people currently rebuild per-project.

Efficiency numbers from the README, for the 4B model:

Inference: on one H20 in BF16, weights occupy 8.6 GiB (4B) / 17.6 GiB (9B). On 10-minute, 2-fps videos, answer-only decoding (no CoT) cuts median post-TTFT latency from 4.78 s to 130 ms and total latency 29.03 → 24.30 s — most of the cost is video prefill, but the interactive feel changes completely.

The one big gap: all benchmark tables in the README are images, so I can't quote per-task numbers. The claim is that Video-ORA-9B "leads the matched seven-family comparison without CoT decoding" against LLaVA-OneVision-2, VideoChat3, and Qwen3-VL (via OneThinker's reported scores), with a footnote admitting some external numbers use different frame/prompt/decoding settings. There are "11 verified evaluations on Papers with Code," which is a decent sign, but verify per-task numbers against specialist models (e.g., dedicated video segmentation/tracking systems) before adopting the recipe — unified models often trail specialists on the hardest families.

Caveats

• Method paper is arXiv 2608.20492; I've only seen the README, so the exact form of the correction term and sign-balancing criterion needs a paper read.
• Training assumes you bring licensed local data; the training set itself doesn't appear to be released.
• Training/eval scripts are dry runs by default — you must inspect the resolved command and add --run. Nice touch, slightly unusual.
• Pinned CUDA stack, veRL + vLLM 0.19.1 + FSDP: multi-GPU territory for training (peak ~51 GB/GPU at 4B even with pruning).

The broader takeaway: this is the video analogue of the growing "expert iteration with ground truth injected" family, but done cleanly — it turns any densely annotated perception dataset into RL supervision without generating CoT traces. If it holds up, it's a very cheap way to get RL benefits on tasks where verifiable rewards exist but positive rollouts are rare.

Try it

MODEL=OraRL/Video-ORA-9B

vllm serve "$MODEL" \
  --served-model-name Video-ORA-9B \
  --trust-remote-code \
  --dtype bfloat16 \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --limit-mm-per-prompt '{"image": 1, "video": 1}'

Set enable_thinking=false in the chat template for the fast answer-only mode. Live demo and project page: https://orarl.github.io/ · weights under the OraRL Hugging Face org.