OneSteer: Unlocking Steerable General Humanoid Motion Tracking via One-Step Diffusion Whole-Body Control

Abstract

Deployment objectives often specify desired outcomes without providing revised trajectories, while parts of the original reference must still be followed. We present OneSteer, a one-step diffusion whole-body controller that combines general motion tracking with selective test-time steering. To accommodate new objectives, differentiable task costs guide optimization of the generator’s input noise with frozen weights, while body-group reference masks relax selected tracking requirements. Predicted state–action plans provide a shared interface for task optimization and feedback-conditioned execution. For nominal tracking, the model trained from scratch achieves the lowest local tracking error among the evaluated baselines on successful held-out AMASS and BONES clips. For steering, one frozen distilled controller adapts across diverse deployment objectives in simulation, raising the clean passage rate under a 1.25 m bar from 2% to 90%. On a physical Unitree G1, OneSteer completes all 27 nominal tracking trials across nine diverse whole-body motion sequences without falling. With a steering cost derived from a signed distance field built using only onboard LiDAR, it also passes under a 1.1 m low bar in four of five attempts and steps over a 12 cm hurdle without falling in all three attempts. Beyond control, prediction variation in the model trained from scratch ranks likely tracking failures without physics rollouts. This scoring method is more than 10× faster than simulation-based scoring with five rollouts per clip and guides targeted fine-tuning.

Running on a Unitree G1

OneSteer tracking reference motions on the real robot, exported to ONNX and run on the onboard Jetson Orin NX at 50 Hz. The overhead gantry is a slack safety line.

Highly dynamic motions

Backflips on the real robot, plain tracking with the same controller.

Steering on the robot

The same frozen policy with a whole-body distance cost from a LiDAR map built before each run and held fixed. It passed under a 1.1 m bar in four of five attempts and stepped over a 12 cm hurdle without falling in all three (once without touching it); the unsteered controller failed every attempt. With a dance reference instead of walking and the same cost, the robot keeps dancing while it passes under the bar (two of four attempts; the large arm swings of the dance disturb its balance while it crouches) and steps over the hurdle (all three attempts). In these trials the steered policy ran on a tethered RTX 4090 computer.

Steering a frozen tracker

Steering bends the plan

Each clip shows the generator’s 0.96 s plan as ghosts. Top: without steering (grey ghosts, red where the plan hits the obstacle; gold is the reference motion). Bottom: the same reference after test-time steering (blue ghosts). The cost moves the plan itself, and the robot executes the moved plan.

Other objectives

Side by side

Every clip is a closed-loop physics rollout. Panels run left to right. The leftmost panel is the reference or the RL teacher. The others are OneSteer, without and with steering.

Terrain is one more input

The generator can take a new condition without changing its structure. We encode a 21×21 height scan around the robot into 36 terrain tokens and append them to the generator’s input sequence. The reference stays a flat-ground motion, the inverse-dynamics head is unchanged, and the model is fine-tuned with RL on stairs, slopes, boxes and rough ground.

Left: the flat-ground reference motion. Right: the terrain-conditioned model on the terrain shown. The label gives the terrain family, the difficulty level (L0–L9) and the final distance to the end of the reference path. These clips are picked from cells the model passes; the table below covers all cells.

88 walking and jogging clips that cross the terrain, each on six terrain families and ten difficulty levels, simulation only. All rows are means of four evaluation passes (the spread of the last column is about 0.02–0.05). “Switched off” keeps the fine-tuned weights and masks the tokens, so the gain comes from the scan. The terrain model starts from the tracker trained from scratch. On flat ground it tracks nearly as before (success 0.963 vs. 0.963, local error 27.6 vs. 26.0 mm), and steering still helps on hurdles, with about half the gain of the model without terrain input. We have not run this model on the real robot.

Plan, then execute

A competitive general tracker

All methods run in one common harness: MuJoCo sim-to-sim, official checkpoints; 150 AMASS windows (held-out subjects) and all 512 BONES windows (unseen actors). Errors are on successful clips. On BONES, 18 windows need stairs, ladders or boxes that the flat-ground harness does not have; every method fails them, so success is capped at 96.5%. Best in bold, second underlined.

The distilled model is on par with its RL teacher. The model trained from scratch with RL, with no teacher, has the lowest local error E_l on both sets, and on BONES it fails only the 18 scene-dependent windows.

Steering inside the generator beats editing the reference

79 walking motions of unseen actors in simulation, three runs per steered condition. The RL teacher tracks a reference edited with the same cost, offline or at every tick (closed loop); J: task-group joints, R: also root height and pitch. OneSteer is steered inside the generator and tracks the original reference. Hurdles: feasible passes may touch the hurdle with a foot, clean passes do not. Latency per control step on one RTX 4090. Bold: best real-time row (95th percentile within the 20 ms period of the 50 Hz loop on the robot’s Orin NX); underlined: best slower row.

The edited reference asks for crouches the deterministic tracker cannot execute, so editing even at every tick stays at 0.11 under a 1.25 m bar, while steering the generator in real time reaches 0.90. A planner followed by a separate tracker also falls short: the teacher tracking OneSteer's own steered plan passes the 1.25 m bar in 0.74 of the trials, so the gain also relies on the action head trained with the generator.

BibTeX

@misc{onesteer2027,
  title     = {OneSteer: Unlocking Steerable General Humanoid Motion Tracking
               via One-Step Diffusion Whole-Body Control},
  author    = {Anonymous},
  year      = {2027}
}