unitree-g1-mujoco

Unitree G1 MuJoCo model for LeRobot

This fork of lerobot/unitree-g1-mujoco defaults to the 23dof (rev_1_0) body with the Pollen Robotics AmazingHand end effector and a D455 pan/tilt head (one published head camera). Body (29dof/23dof), end effector (rubber_hand, none, dex1, dex3, amazing_hand), head mount (fixed/pan_tilt) and head sensor (d435i/d455) combine freely: sim/mjcf/compose.py composes the selected scene at load time from a bare body and reusable parts files. This is the model repository loaded dynamically by LeRobot through env.py.

The existing make_env() entry point, body motor commands, body state DDS topics, simulation stepping and ZMQ image message format are preserved.

Files. Every model is one bare body plus parts files:

An end-effector parts file defines left_ee_mount / right_ee_mount bodies in the 29dof wrist_yaw_link frame, with the meshes, defaults, actuators, sensors and equality constraints they use; a head mount defines a body tree in the 23dof torso_link frame with a head_sensor_mount, and a sensor file a head_sensor body in that mount's frame. The extracted files name their source; Unitree's full models are in this repo's history (commit 1801640). Generated files are checked by ToolsTests and must not be edited by hand.

Body variants

config.yaml defaults to BODY: 23dof, END_EFFECTOR: amazing_hand and HEAD_MOUNT: pan_tilt, HEAD_SENSOR: d455. The 23dof body models the rev_1_0 hardware, which has no waist_roll/waist_pitch and no wrist_pitch/wrist_yaw per arm (6 fewer joints than the 29dof body).

The DDS LowState_/LowCmd_ messages always carry 35 motor slots. The 23dof body publishes only slots 0-12, 15-19, 22-26; the 6 slots it has no joint for (13, 14, 20, 21, 27, 28, i.e. waist_roll/waist_pitch and each arm's wrist_pitch/wrist_yaw) are left at the SDK default q=dq=tau=0, and any command written to them is ignored. This mapping lives in sim/unitree_sdk2py_bridge.py's JOINT_SLOTS.

To use the 23dof body from LeRobot:

lerobot-record \
  --robot.type=unitree_g1 \
  --robot.is_simulation=true \
  --robot.sim_env_repo_id=k-valentin/unitree-g1-mujoco \
  ...

Direct users of this repository select it with make_env(body="23dof") (or BODY: 23dof in config.yaml).

Choose the end effectors

End effectors are named after the hardware; each one mounts on either body. config.yaml defaults to END_EFFECTOR: amazing_hand. Finger counts, effort limits and the published camera list follow the selection automatically.

HEAD_MOUNT: pan_tilt adds the 2-DoF pan/tilt head (ZMQ bridge) to any of them, carrying any HEAD_SENSOR.

Mounting. The *_ee_mount bodies go inside *_wrist_yaw_link on the 29dof body, and inside *_wrist_roll_link at the wrist pitch+yaw offsets (+0.084 m) on the 23dof body, i.e. where the hand sits on a 29dof arm with its wrist pitch/yaw at zero. The bodies carry the bare wrist inertia: on 29dof the URDF's bare wrist_yaw_link; on 23dof Unitree's fused wrist-roll + rubber-hand link minus the rubber hand's mass (Unitree lumps it into the 29dof wrist_yaw_link, which gives it). Dex hands on the 23dof body and the AmazingHand on either body have no adapter CAD yet, so those mounts are placeholders. The composed MJCF is written to a temp directory (<tmp>/unitree_g1_mujoco/) with absolute mesh paths.

Direct users of this repository can also call:

from env import make_env

if __name__ == "__main__":
    env = make_env(end_effector="dex3")  # Omit the argument for dex1.
    try:
        env.reset()
        while True:
            env.step()  # Body commands continue to arrive over DDS.
    finally:
        env.close()

Pan/tilt head and AmazingHand

HEAD_MOUNT: pan_tilt adds a 2-DoF Dynamixel pan/tilt head (assets/heads/pan_tilt.xml), here with HEAD_SENSOR: d455, an Intel RealSense D455 (assets/sensors/d455.xml), and END_EFFECTOR: amazing_hand a Pollen Robotics AmazingHand on each wrist (assets/end_effectors/amazing_hand.xml). config.yaml defaults to BODY: 23dof, END_EFFECTOR: amazing_hand, HEAD_MOUNT: pan_tilt, HEAD_SENSOR: d455, CAMERAS: ["head_camera"].

• Head = mount + sensor: HEAD_MOUNT and HEAD_SENSOR are independent. The fixed mount is the body's own head_sensor_mount (the stock head, at the pose of the stock camera); pan_tilt replaces it with assets/heads/pan_tilt.xml. A sensor (assets/sensors/<name>.xml) is a head_sensor body placed in the mount's head_sensor_mount, looking along its +x with +z up, holding a head_camera and optionally its own geoms/inertia. A new camera is a new sensors file and its name in sim/mjcf/compose.py's HEAD_SENSORS.

• Camera intrinsics: each sensor's head_camera carries pinhole intrinsics for the recorded 640x480 color mode (resolution, focalpixel, principalpixel, sensorsize), so a 640x480 render has the real camera's geometry. They come from the Intel D400-series datasheet until each unit's calibration is read (rs-enumerate-devices -c); the derivation is in each sensor file. Lens distortion is not modelled.

  HEAD_SENSOR	Datasheet RGB FOV (H x V)	640x480 focal length	640x480 FOV (H x V)
  d435i (default, stock G1)	69 x 42 deg (1920x1080)	623.01 px	54.4 x 42.1 deg
  d455	90 x 65 deg (1280x800)	380.36 px	80.1 x 64.5 deg

• Mount chain: torso_link -> head_servo_link (fixed) -> xl330_link (head_yaw_joint, pan, range -0.7..0.7 rad) -> head_tilt_link (head_pitch_joint, tilt, range -1.5708..0.8 rad) -> head_sensor_mount (fixed, the front face of the tilt bracket's tip). Both joints are position actuators with kp=5.

• Head meshes: the links are primitive boxes (head_servo_link 20x20x20 mm, xl330_link 20x34x26 mm, head_tilt_link 90x25x25 mm); config.yaml's HEAD_MESHES: true (default) swaps in the lab's assets/meshes/head/{head_servo_link,xl330_link,head_tilt_link}.STL when all three are present (see THIRD_PARTY_NOTICES.md for their provenance).

• Hands: each AmazingHand contributes 8 actuated hinge joints named by servo ID (left_hand_motor11_joint .. left_hand_motor18_joint, right_hand_motor1_joint .. right_hand_motor8_joint; finger 1 servo 1 = the lowest ID), ctrlrange snapped to exactly +-pi/2.

• Joint names vs lerobot: head/hand joints and actuators use the G1 MJCF snake_case style, and sim/head_hand_sim.py's mjcf_joint_name maps lerobot's Unitree-style motor names onto them: kHeadYaw -> head_yaw_joint, kHeadPitch -> head_pitch_joint, kLeftHandMotor11 -> left_hand_motor11_joint. Link names (head_servo_link, xl330_link) keep the lab URDF's part names; its d455_link is head_tilt_link here, as it can carry any sensor. Every body/joint/geom/mesh/material/actuator from the source onshape-to-robot export is prefixed left_hand_/right_hand_, including the 24 passive ball/hinge joints per hand that close the finger's parallel four-bar linkage (<equality connect> constraints, carried over renamed). All hand geoms are visual-only (contype="0" conaffinity="0", no collision geoms in the source export). Meshes are copied from AHSimulation/AH_Left|Right/mjcf under assets/amazing_hand/{left,right}/; see THIRD_PARTY_NOTICES.md for the CC-BY-4.0 attribution.

• Mount transform: the Onshape connector transform from the G1 wrist to the AmazingHand mount is not available yet, so tools/build_amazing_hand.py uses a placeholder (pos="0.13 0 0", a quaternion rotating the hand's local +z, its finger-reach axis, onto the wrist's local +x, so the hand extends away from the elbow). Replace both the offset and the rotation once the CAD is available.

• DDS scope: NUM_MOTORS/motor_effort_limit_list stay body-only (23, matching JOINT_SLOTS["23dof"]); the D455 pan/tilt head and the 16 hand actuators are never wired to DDS. sim/base_sim.py's joint scan never adds joints to left_hand_index/right_hand_index when END_EFFECTOR == "amazing_hand" (is_dds_hand is False), so those two lists stay empty and dds_actuator_index covers only the 23 body actuators; the 24 passive linkage joints per hand also carry a left_hand_/right_hand_ prefix but are never actuated, so they would not match this scan even if it ran. mj_data.ctrl for the 18 non-DDS actuators (head + both hands) is left at whatever another writer sets, never overwritten by the body torque loop -- that other writer is the ZMQ bridge described next.

• Head/hand ZMQ bridge: whenever HEAD_MOUNT != "fixed" or END_EFFECTOR == "amazing_hand", env.py's make_env() starts sim/head_hand_sim.py's SimHeadHandDevice (a MuJoCo-backed stand-in for lerobot's real HeadHandDevice) behind lerobot.robots.unitree_g1.headhand_zmq.HeadHandServer, in a daemon thread bound to 127.0.0.1. Ports default to HEADHAND_STATE_PORT: 6003 / HEADHAND_CMD_PORT: 6002 in config.yaml, overridable with UNITREE_G1_MUJOCO_HEADHAND_STATE_PORT / UNITREE_G1_MUJOCO_HEADHAND_CMD_PORT. A lerobot.robots.unitree_g1.UnitreeG1 robot with is_simulation=True, head_mount=pan_tilt and end_effector=amazing_hand talks to this bridge exactly as it would to real Dynamixel/Feetech hardware, and passes its body/end_effector/head_mount/head_sensor to make_env as the UNITREE_G1_MUJOCO_* variables below. The server thread is stopped from env.close().

Headless / option overrides

LeRobot's make_env(repo_id) cannot forward keyword options, so every make_env option can also be set through the environment as UNITREE_G1_MUJOCO_<OPTION>, e.g. UNITREE_G1_MUJOCO_ONSCREEN=0 (no viewer window), UNITREE_G1_MUJOCO_PUBLISH_IMAGES=0, UNITREE_G1_MUJOCO_CAMERA_PORT=5556, UNITREE_G1_MUJOCO_BODY=29dof, UNITREE_G1_MUJOCO_END_EFFECTOR=dex1, UNITREE_G1_MUJOCO_JOYSTICK_TYPE=xbox. Keyword arguments still win when calling make_env directly. Note that after disconnect() the Python process may not exit on its own because of CycloneDDS finalizers; end the session with Ctrl-C.

Keyboard (viewer window)

Key presses in the MuJoCo viewer window are forwarded to LeRobot's unitree_g1_keyboard teleoperator (sim/keyboard_forward.py), so it works on Wayland desktops where pynput cannot capture keystrokes: focus the viewer window and use its key map (arrows = head, q/e = hands, w a s d / i j k l = sticks, t/g = base height). Keys 7/8/9 still drive the elastic band.

Gamepad

The bridge reads a pygame joystick and publishes it as the robot's wireless_remote (JOYSTICK_TYPE in config.yaml, default dualshock4 for a Sony DualShock 4 under SDL's HIDAPI driver; xbox and switch as upstream). Plug the pad in before launching.

Cameras

All three streams are enabled by default on tcp://127.0.0.1:5555, with 640 × 480 images and the existing approximately 30 Hz publishing setting.

Each stream is advertised through the same top-level JPEG key and nested images / timestamps entries as head_camera. The existing view_cameras_live.py discovers the names from those messages. They are also listed in env.camera_configs, env.camera_names and env.metadata["cameras"]. The wrist cameras move with their respective wrists, with 95° vertical field of view and the approximate extrinsics from the HIW-500 model builder.

LeRobot's ZMQ cameras require explicit client configuration, just as the head camera does. Pass this dictionary as UnitreeG1Config(..., cameras=cameras):

from lerobot.cameras.zmq.configuration_zmq import ZMQCameraConfig

cameras = {
    name: ZMQCameraConfig(
        server_address="127.0.0.1", port=5555, camera_name=name,
        width=640, height=480, fps=30,
    )
    for name in ("head_camera", "left_wrist_cam", "right_wrist_cam")
}

An equivalent camera configuration is in lerobot_cameras.json. make_env(cameras=["head_camera"]) selects a subset; publish_images=False disables publishing. onscreen=False disables the viewer for headless use.

Gripper control

Dex1 fingers use force actuators, driven by the existing bridge's external PD controller. They start open at 0.0245 m. To preserve the simulator's hand transport, the first two motor entries on rt/dex3/left/cmd and rt/dex3/right/cmd command fingers 1 and 2, with matching state topics. Gripper q is in metres and tau is in newtons; each finger is limited to 20 N. Command both fingers to the same position for symmetric opening or closing. Original hand mode retains all seven rotational command entries per side.

The simulation lower limit is -0.023 m, following HIW-500's mesh closure trim. The source URDF retains the official -0.020 m lower limit. Run python tools/build_dex1_model.py --official-limits to use that limit in MJCF too; it leaves approximately 5.88 mm between the supplied finger pads. This model update does not add finger actions to LeRobot's 29-body-motor UnitreeG1 action schema.

Development and checks

Install the existing Unitree SDK2 / CycloneDDS prerequisites, then python -m pip install -r requirements.txt. Regenerate the generated parts files with python tools/build_dex1_model.py and python tools/build_amazing_hand.py.

MUJOCO_GL=egl python -m unittest discover -s tests -v
MUJOCO_GL=egl python tests/smoke_live.py
MUJOCO_GL=egl python tests/smoke_live.py --end-effector dex3

The regression suite uses real MuJoCo for both variants, motor/observation mapping, force-controlled closure, wrist camera motion, rendering and camera publisher shared-memory buffers. It isolates DDS with a test double. smoke_live.py additionally requires the real Gymnasium, Unitree SDK2 and ZMQ/OpenCV dependencies and checks the live environment and transports. See VALIDATION.md for what was executed for this update.

Sources

• Base model/runtime: lerobot/unitree-g1-mujoco.
• Dex1-1 URDF, meshes and wrist-camera geometry: Hxxxz0/HIW-500-controoler.
• LeRobot loading and camera configuration checked against main at b6ec006.

The original Dex1 source URDF, Apache 2.0 license and attribution notice are in reference/hiw500/. Unitree mesh assets retain their upstream terms.