lzyang2000/perceptive_cbf_rl

TL;DR: PAC-MAN (Caltech AMBER Lab) trains a Unitree G1 humanoid to dodge hand-thrown balls using only a head-mounted depth camera, and deploys it zero-shot on hardware: 19/20 throws dodged, zero falls. The two ideas worth stealing are (1) using control barrier functions as training-time reward shaping for whole-body safety rather than as a runtime filter, and (2) designing the sim observation to be exactly what the real perception stack can produce — a tiny 16×9 ball-only masked depth image — so the sim-to-real gap collapses to a single, controllable layer.

What it is

A full training pipeline, simulation benchmark, and hardware deployment stack for perceptive whole-body evasion on the G1. The policy is PPO with an adversarial motion prior (AMP, trained on ~100 s of retargeted human mocap: walks, runs, standing dodges and leaps) plus CBF-derived safety rewards. Two barrier formulations are studied: a per-link clearance CBF (Link-CBF — the deployed configuration) and a joint-space CBF used as reward guidance, with an optional privileged runtime projection kept as a simulation-only ceiling. Emergent behaviors include ducking, sidesteps, and step-overs, selected by the policy depending on ball trajectory (descending across the legs vs. rising to torso/head height).

The CBF angle is the genuinely interesting research move. CBFs are usually deployed as online safety filters wrapping a nominal controller, which is hard for high-DOF humanoids with vision inputs. Here the barrier conditions are baked into the reward, so the safety structure shapes what the policy learns rather than clipping what it outputs. The README's seeded-throw comparison shows this matters: identical throws are hit under a plain distance reward but dodged under Link-CBF, and hit with a fixed camera but dodged with a gimbal — i.e., both the barrier design and the perception regime independently decide outcomes.

The perception trick

Rather than feeding raw depth and hoping domain randomization covers the gap, the policy only ever sees a ball-segmented, masked depth image downsampled to 16×9 with a sparse frame stack. On hardware, a ZED camera plus EfficientTAM produces exactly that representation. Sim-to-real then reduces to "does the segmenter find the ball," which is a much easier problem than photorealistic depth simulation. This is a clean, reusable recipe for perceptive RL deployment.

What's actually released

Notably complete: training code (mjlab/MuJoCo Warp, AMP adapted from AMP_mjlab, rsl_rl PPO), G1 MJCF assets, retargeted motion clips and the retargeting recipe from NVIDIA's SEED mocap, a dodge benchmark script, per-experiment launchers for every paper cell, a pytest suite for the CBF terms, and — unusually — the full hardware stack: deployed ONNX checkpoints (dodge_link_cbf.onnx, plus a walk policy with automatic mode switching), the ZED+EfficientTAM camera node, and a Unitree DDS bridge, all as three processes over localhost UDP. MIT license (robot model and motion clips keep upstream licenses).

Evidence: the 19/20 hand-throw hardware result with 0 falls, the seeded ablations above, and a video/live demo on the project page. No throw-speed/distance stats are in the README — check the paper before judging how adversarial the evaluation is. Training covers exactly two threat types (descending across legs, low-arc rising), so generalization to arbitrary projectiles, multiple balls, or throws from unseen angles is untested here.

Caveats

Linux x86_64 + NVIDIA GPU required; the deploy stack is a separate Python 3.8 project needing pyzed, EfficientTAM weights, and the Unitree SDK — real friction, but documented in deploy/SETUP.md with a sim-to-sim validation path. The deployed policy uses the fixed camera; the gimbal variant that dodges more throws in sim uses an oracle-aimed pitch, which is a privileged signal. The Joint-CBF "+filter" configuration is explicitly a simulation ceiling, not deployable. Hand throws by (presumably cooperative) experimenters are a softer test than a pitching machine.

Try it

git clone https://github.com/lzyang2000/perceptive_cbf_rl && cd perceptive_cbf_rl
uv sync                                # Python 3.11 + mjlab + torch (cu128)
uv run python scripts/list_envs.py     # then scripts/train.py / play.py / dodge_benchmark.py

No robot needed to train and evaluate in sim; the live demo and video show the hardware behavior.