Overview / Software
Software

Stack. Setup. Run.

One command interface for teleoperation and learning, with shared retargeting and control.

01 · Stack

The software stack.

Command sources share the server and controller. The bridge owns the plant; ROS 2 consumes state and supplies navigation goals.

Retargeting

Retargeting.

Torso-frame estimation, shoulder-relative reach and palm-frame finger targets.

Paper equations
Paper · operator state, body and neck
[Δr; Δz] = (wRb)T (wph − wph,a) Δrlean = clipB(Δr),   Δrwalk = Δr − Δrlean B = [−0.02, 0.18] × [−0.12, 0.12] mforward/lateral comfort box qtorso* = qtorso,0 − k Δxlean zlift* = zlift,a + Δz orbase* = orbase,a + Δrwalk qneck* = ypr((wRb)T wRh) − [0, qtorso*, 0]T

wRb = Rz(ψ) is the estimated gravity-aligned body-heading frame. The bounded displacement becomes torso lean; the residual becomes walking intent. Neck angles reproduce head orientation relative to that heading, with torso pitch removed. Subscript a marks a latched anchor; * marks a reference.

Arm and finger retargeting objectives
rpeᵢ* = ror,i + si (wRb)T (wpi − woh,i),   si = Lir/Lih qIK = arg minq ‖rpeᵢ(q) − rpeᵢ*‖² + wdir Σj∈U(1 − uj(q)Tuj*) + wrest‖q − qrest‖² subject to qmin ≤ q ≤ qmax,   q ∈ R⁸ per arm and wrist g* = arg ming ‖P(g) − P̂‖F² + wpinch Σm‖δm(g) − δ̂m‖² subject to gmin ≤ g ≤ gmax,   g ∈ R¹⁶ per hand

Arm IK matches palm position and palm-normal, finger-axis and elbow directions, using warm-started damped Gauss–Newton. Finger retargeting matches fingertip positions P and thumb-to-fingertip vectors δ in the palm frame. The wrist is already part of the eight-coordinate arm solve.

Paper: Retargeting →
Motion model

Motion model.

Raise the lift and compare acceleration, braking and measured tilt in MuJoCo.

Paper equations
Paper · three-module swerve and height-aware limits
vj = v + ω S rj,   S = [0 −1; 1 0],   j ∈ {1,2,3} ac ≈ v̇ + ω × v + ω̇ × rc − ω²rc hc(zlift) = h0 + kh zliftCoM height fit from the model akmax = g ℓk / [σk hc(zlift)]edge-wise acceleration bound u = maxk (−nkT ac / akmax) ≤ 1scale the base command to stay within the envelope

The base-frame twist is νb = (v, ω). Module position rj is measured from the center of rotation; its velocity direction sets steering and its magnitude sets drive speed, scaled by cos ej for steering error. Each support edge has outward normal nk, CoM lever arm ℓk, and safety factor σk ≥ 1. The approximation fixes the planar CoM offset rc and neglects upper-body CoM motion.

Paper: Motion Model →
Whole-body control

Shared control.

Try joint targets, task-space targets and external forces. Compare impedance with gravity compensation.

Watch the real-robot compliance test ↗

Paper equations
Paper · collision avoidance and compliant control
q̇* = arg minq̇ ½ ‖q̇ − q̇des‖² q̇des = (qIK − q) / ΔtIK reference → desired velocity q̇min ≤ q̇ ≤ q̇maxjoint and velocity limits −Jc q̇ ≤ κ (dc − ds), ∀ c ∈ Ccapsule-clearance constraint τ = τg(q) + τf(q̇) + K(q* − q) + D(q̇* − q̇)

The paper projects the two arm-and-wrist IK solutions (16 coordinates) onto the nearest admissible joint velocity. Jc = ∂dc/∂q is the clearance Jacobian for capsule pair c, ds is the margin, and κ sets the approach-rate bound. The arms and torso then track references with gravity and friction compensation plus low-gain impedance; hands, neck and lift use separate position control.

Paper: Whole-Body Teleoperation and Control →
02 · Setup

Setup TUI & Studio.

Inspect modules, review joint calibration and check live feedback in the terminal or browser.

SIMULATION · SCRATCH CALIBRATION
Recorded workflow

Setup in the terminal

Module → joint → calibration. The terminal and Studio use the same setup service.

SIMULATION SETUP
Recorded workflow

Setup Studio

The same modules and calibration windows, with an interactive robot view.

python3 setup/tui.py --sim --no-takeover
python3 setup/studio.py --sim --no-takeover
03 · Server bring-up

Bring up the server.

MILO’s launchers open the cockpit: robot state, module status, controls and logs in one terminal.

LIVE MUJOCO · RECORDED SESSION
Recorded workflow

The server cockpit

An actual terminal capture attached to the isolated MuJoCo instance.

Simulation

./sim_run.sh --home

Real robot

./real_run.sh

The recording uses a separate simulation instance. Real hardware setup and motor calibration require an operator at the robot.

Claude Code onboarding walkthrough →