lightorigins/LightNav-0

TL;DR: LightNav-0 is a 4B-parameter generalist navigation policy from the Light Origins team that turns Qwen3-VL into a single model for instruction-following (VLN), open-vocabulary object-goal navigation, and visual tracking — no waypoint predictor, no task heads, no per-embodiment experts, just an extended vocabulary. It claims SOTA on essentially every monocular navigation benchmark from a single shared checkpoint, ships Apache-2.0 code, weights, a WebSocket serving stack, and ROS 2 deployment adapters, and its scaling analysis has one genuinely useful finding: training-environment diversity is the lever that keeps paying, not parameters or data hours.

The idea

The interesting move is architectural restraint. Rather than bolting a diffusion policy or waypoint head onto a VLM, LightNav-0 extends only the token vocabulary and decodes everything through the backbone's original autoregressive LM head:

• Dual-channel pointing tokens: at each decision step, the model first emits an affordance point (a feasible free-space direction) and an object point (the goal), each as one image-grid token. This is an explicit spatial reasoning trace in pixel space, produced before any motor output — a chain-of-thought that's grounded rather than verbal.
• Three RVQ action tokens: a residual vector-quantized tokenizer maps 10 future SE(2) waypoints to a coarse 256-entry codebook (~0.9 m resolution) plus two residual codebooks (~7 cm, ~4 cm). Any prefix already decodes to an executable coarse trajectory; each extra token refines geometry. Three tokens per decision is remarkably cheap.

Task identity comes only from the instruction — there's no task token — and the SE(2) waypoint output is embodiment-agnostic, handed to each robot's own low-level controller. That's how one checkpoint drives humanoids, quadrupeds, wheeled bases, and drones.

A second nice trick: history compression shaped like the Ebbinghaus forgetting curve — sampling rate decays exponentially with frame age while pooling stride grows, so old frames contribute few coarse tokens and the current frame stays sharp, bounding context length without a fixed-resolution summary.

Evidence

One checkpoint, no per-benchmark fine-tuning, monocular RGB only. R2R val-unseen 68.5 SR / 62.8 SPL and RxR 73.6 / 64.5, edging out Qwen-RobotNav-8B and CorrectNav. ObjectNav is the strongest showing: HM3D v2 at 79.5 SR and HM3D-OVON unseen categories at 47.0 vs. MTU3D's 40.8. On EVT-Bench distractor tracking it posts 82.6 SR — nine points over ReferTrack and reportedly above every panoramic multi-camera system in the paper.

Their own INSIGHT-Bench (1,097 episodes, 210 indoor/outdoor scenes, shared 120° RGB interface) shows everyone struggling but LightNav-0 struggling least:

The scaling analysis is the takeaway worth remembering: model scaling saturates at 4B (8B is mixed-to-worse), data scaling plateaus (the last corpus doubling buys 0.8 SR), but environment diversity keeps paying — 1/8 → full environment coverage adds 16.7 R2R SR and 21.1 RxR SR. If it replicates, this says invest in scene coverage, not GPU-hours or parameters, echoing what manipulation folks have found about task diversity.

Caveats

• All headline numbers are simulation (Habitat/VLN-CE/EVT-Bench). The real-robot evidence is the demo video and a deployment stack; there's no quantified real-world success rate in the README. The triage note about verifying weight downloads on the HF repo is worth heeding — this is a 4-day-old repo from a team without a public track record.
• INSIGHT-Bench is self-authored, and the ~16-point margin there is the most dramatic result. Independent runs would be needed before treating it as a gap in kind.
• No latency/throughput numbers, which matter for tracking a moving person from a 4 fps stream through a 4B model.
• "Baselines are the strongest monocular entries" — a fair framing, but check the paper's full tables for the panoramic comparisons before quoting SOTA.

Trying it

Python 3.11, vLLM backend, model from HF; a self-contained MuJoCo TurtleBot demo needs no GPU on the client side:

git clone https://github.com/lightorigins/LightNav-0.git && cd LightNav-0
python3.11 -m venv .venv && source .venv/bin/activate
pip install -e ".[vllm,video]"
hf download LightOriginsHQ/LightNav-0 --local-dir checkpoints/LightNav-0

lightnav-predict --model_path checkpoints/LightNav-0 \
  --backend vllm_local --video clip.mp4 --fps 4 \
  --instruction "follow the person in the red shirt"

# or the browser-based MuJoCo demo:
cd mujoco_demo && ./run.sh   # then open http://127.0.0.1:8088

robot_deploy/ has ROS 2 adapters for Unitree Go2 and LimX TRON 1, plus a bring-your-own-robot interface. Model card: huggingface.co/LightOriginsHQ/LightNav-0.