microduck-3d-bipedal-teleop

🦆 MicroDuck 14-DOF Bipedal & Skating // Live 3D Teleoperation Cockpit & Physical AI Benchmark

High-Precision 14-DOF Bipedal & Skating Duck Robot Reinforcement Learning Policies, Sim-to-Real BAM Actuator Dynamics, 50Hz Teleoperation Dataset, and Interactive Three.js WebGL Real-time 3D Teleoperation Cockpit.
🌐 English Documentation | 🇰🇷 한국어 매뉴얼

[!TIP] 🎮 Real-time 3D Web Teleoperation Space: 🤗 hwihwalab/microduck-3d-bipedal-teleop
📦 Official Model Hub: 🤗 hwihwalab/microduck-3d-bipedal-teleop
💻 Local Execution: Single-click launch via .\run_web.ps1 (Standalone Web 3D App) or .\run_simulator.ps1 (Native MuJoCo C++ Viewer).

<div align="center"> <table width="100%" style="border: none;"> <tr style="border: none;"> <td width="50%" align="center" style="border: none; padding: 6px;"> <a href="https://huggingface.co/hwihwalab/microduck-3d-bipedal-teleop/resolve/main/assets/microduck_bipedal_walking.gif" target="_blank" title="Click to view full resolution GIF"> <img src="assets/microduck_bipedal_walking.gif" alt="MicroDuck 14-DOF Bipedal Walking Simulation" width="100%" style="border-radius: 8px; border: 1px solid #334460; box-shadow: 0 4px 20px rgba(0,0,0,0.5); cursor: zoom-in;" /> </a> <p><em>Figure 1-A: 14-DOF Bipedal Walking Locomotion (50Hz ONNX / BAM 7.4V)</em></p> </td> <td width="50%" align="center" style="border: none; padding: 6px;"> <a href="https://huggingface.co/hwihwalab/microduck-3d-bipedal-teleop/resolve/main/assets/microduck_roller_skating.gif" target="_blank" title="Click to view full resolution GIF"> <img src="assets/microduck_roller_skating.gif" alt="MicroDuck 14-DOF Roller Skating Simulation" width="100%" style="border-radius: 8px; border: 1px solid #334460; box-shadow: 0 4px 20px rgba(0,0,0,0.5); cursor: zoom-in;" /> </a> <p><em>Figure 1-B: High-Speed Roller-Skating & Gliding on Passive Wheels</em></p> </td> </tr> </table> </div>

---

⚡ Key Engineering Highlights & Novelties

• BAM 1.0.1 Sim-to-Real Dynamic Solvers: Recreates realistic voltage-drop resistance, back-EMF, and non-linear Coulomb friction loss of 14x Dynamixel XL330-M6 servos running on a 2S 7.4V LiPo battery (6.5V ~ 8.2V).
• Actuator Latency Buffering: Enforces randomized $15\text{ms} \sim 30\text{ms}$ (delay_lag = 3 ~ 6 steps) communications delay during training, preventing real-world servo chattering and high-frequency resonance.
• Massive 196.6M Experience Scale: $4,096\text{ envs} \times 24\text{ steps} \times 2,000\text{ iterations} = \mathbf{196,608,000\text{ steps}}$, equivalent to 1,092 hours (over 45 days) of bipedal walking compressed into ~35 minutes on massively parallel GPU cluster.
• Dual-Track Operational Architecture:
  • Track 1️⃣ (Web 3D Cockpit): FastAPI + WebSockets (40 Hz telemetry) + Three.js WebGL binary mesh streaming directly from compiled MuJoCo memory buffers.
  • Track 2️⃣ (C++ Native Simulator): MuJoCo 3.12 passive OpenGL viewer with a single-line live terminal HUD (msvcrt Windows polling).
• 3-Speed Gait Transmission: On-the-fly velocity limit switching (SLOW 0.10 / NORM 0.20 / TURBO 0.35 m/s).
• Real-Time Dual Ankle Ground Contact Sensing: Continuous 50 Hz Z-coordinate tracking of left & right foot contact states (● STANCE vs ○ SWING).

---

🏛️ System Architecture & Real-Time Dataflow

The system connects the 50Hz ONNX Reinforcement Learning Policy Loop, MuJoCo BAM Sim-to-Real Actuator Dynamics, and the Three.js WebGL 3D Digital Twin Cockpit into an ultra-low-latency real-time pipeline:

flowchart TD
    subgraph Client_Layer ["1️⃣ Live 3D Web Teleoperation Cockpit (Three.js WebGL)"]
        UI1["Three.js 3D Duck Mesh Renderer"]
        UI2["Real-Time HUD Telemetry (Velocities, Gear, Dual Foot Contact, Power)"]
        UI3["Interactive Teleoperation Controls (Keyboard Shortcuts & 3-Speed Gear)"]
    end

    subgraph Server_Layer ["2️⃣ High-Throughput Streaming Server (FastAPI)"]
        WS["WebSocket Bidirectional Channel (/ws/teleop @ 50Hz)"]
        SRV["Telemetry Serializer & Command Dispatcher"]
    end

    subgraph AI_Engine ["3️⃣ Physical AI Policy Inference Engine (ONNX Runtime)"]
        OBS["61D Unified Observation Vector Builder"]
        POL["9 Core ONNX Reinforcement Learning Policies (50Hz Loop)"]
        ACT["14D Target Servo Positions (Default Pose + Action Scaling)"]
    end

    subgraph Physics_Layer ["4️⃣ Sim-to-Real Physics Engine (MuJoCo 3.12 & BAM)"]
        MJ["MuJoCo 200Hz Multibody Dynamics Solver (dt=0.005s)"]
        BAM["BAM 1.0.1 Dynamixel XL330-M6 Voltage-Drop & Friction Model"]
        SEN["Left & Right Ankle Ground Contact & Battery Power Sensor"]
    end

    subgraph Dataset_Pipeline ["5️⃣ 50Hz Teleoperation Dataset Pipeline"]
        LOG["High-Precision 50Hz Telemetry Stream Logger"]
        PQ["train.parquet (50 Episodes / 12,500 Timesteps)"]
    end

    UI3 -->|"Control Commands (Twist, Posture Flags)"| WS
    WS --> SRV
    SRV --> OBS
    OBS -->|"61D State Vector"| POL
    POL -->|"14D Joint Action Offsets"| ACT
    ACT -->|"14 Servo Target Positions"| BAM
    BAM -->|"Simulated Motor Torques & Voltage Sag"| MJ
    MJ --> SEN
    SEN -->|"State Feedback (Gyro, Joint Pos/Vel, Foot Contact)"| OBS
    SEN -->|"50Hz Live Telemetry Data Packet"| SRV
    SRV -->|"JSON Broadcast Stream"| WS
    WS --> UI1 & UI2
    SEN & ACT --> LOG --> PQ

---

📁 Repository Contents

• microduck-3d-bipedal-teleop.zip: Standalone One-Click 3D Web Cockpit & Simulator Production Archive (47.2 MB ⬇️ Direct Download).
• README.md: Official Model Card & 14-DOF Bipedal Locomotion Research Specifications.
• README_KR.md: Comprehensive Korean Manual & Teleoperation Guide.
• experiment_results.json: 4,096 Parallel Environments Training Metrics & Benchmark Results.
• checkpoints/model_final.pt: 2,000 PPO Iterations (196.6M Timesteps) PyTorch Actor-Critic Weights.
• policies/: 9 Core 50Hz ONNX Policy Weights (alpha_walking, alpha_stand, alpha_sitstand, alpha_ground_pick, roulade, ball_kick_left/right, roller, roller_crouch).
• LICENSE: Apache-2.0 License.

---

🐍 Quickstart: Python ONNX Standalone Inference

You can run any of the 9 policy ONNX models directly with onnxruntime and numpy:

import onnxruntime as ort
import numpy as np

# 1. Load the 14-DOF Bipedal Walking ONNX Policy
session = ort.InferenceSession("policies/alpha_walking.onnx")
input_name = session.get_inputs()[0].name  # 'obs'

# 2. Construct the 61-dimensional Unified Observation Vector (Batch Size = 1)
obs = np.zeros((1, 61), dtype=np.float32)

# Example: Command forward walking at 0.20 m/s (NORM gear)
obs[0, 3:6] = [0.0, 0.0, -1.0]  # Projected gravity vector (pointing down)
obs[0, 48]  = 0.20             # cmd_vx = 0.20 m/s
obs[0, 49]  = 0.00             # cmd_vy = 0.00 m/s
obs[0, 50]  = 0.00             # cmd_wz = 0.00 rad/s

# 3. Compute 50 Hz Target Joint Action Offsets (14D)
action = session.run(None, {input_name: obs})[0]
print("Computed 14D Action Offsets (rad):", action.squeeze())

# 4. Decode to physical Dynamixel Motor Positions
DEFAULT_POSE = np.array([
    0.0, -0.0873, -0.4579, -0.0049,  0.4530,  # Left Leg:  yaw, roll, pitch, knee, ankle
    0.3491,  0.3491,  0.0,     0.0,            # Neck/Head: neck_pitch, head_pitch, head_yaw, head_roll
    0.0,  0.0873,  0.4579,  0.0049, -0.4530   # Right Leg: yaw, roll, pitch, knee, ankle
], dtype=np.float32)

ACTION_SCALE = 0.25
target_motor_positions = DEFAULT_POSE + (action.squeeze() * ACTION_SCALE)
print("Target Dynamixel Angles (rad):", target_motor_positions)

---

🦾 14-DOF Robot Kinematics & Motor ID Mapping

The MicroDuck robot features 14 high-precision Robotis Dynamixel XL330-M6 smart servos. Target actions are indexed in the following strict order:

[!NOTE] Center of Mass (CoM) Forward Shift (STAND2 Keyframe): The trunk is intentionally pitched and shifted $\sim 5\text{mm}$ forward over the ankle rotation axis. This ensures the robot's center of mass sits securely inside the foot support polygon, preventing backward toppling without needing counterweight head drooping.

---

🔬 MicroDuck Sim-to-Real Research Specifications

1. 61D Unified Observation & 13D Command Vector Breakdown

All policies operate on a unified 61-dimensional state representation:

[obs_00 : obs_03] ( 3) -> Base Angular Velocity (Roll, Pitch, Yaw gyro [rad/s])
[obs_03 : obs_06] ( 3) -> Projected Gravity Vector (gx, gy, gz)
[obs_06 : obs_20] (14) -> Joint Positions Error (current_pos - DEFAULT_POSE [rad])
[obs_20 : obs_34] (14) -> Joint Velocities (14 joint angular rates [rad/s])
[obs_34 : obs_48] (14) -> Last Applied Action (t-1 action offsets)
[obs_48 : obs_61] (13) -> Unified Command Vector (Twist [3], Head [4], Body [4], Flags [2])
--------------------------------------------------------------------------------
Total Dimension = 61 float32 values

---

2. BAM (Better Actuator Models) Dynamic Parameters

• Actuator Hardware: 14x Robotis Dynamixel XL330-M6
• Nominal Battery Voltage: 7.4V (2S LiPo, operating range 6.5V ~ 8.2V)
• Peak Torque Limit: $0.963\text{ N}\cdot\text{m}$ (stall current $\approx 1.85\text{A}$)
• Firmware Stiffness: $K_{p,\text{fw}} = 200.0$ (internal Dynamixel PID loop modeling)
• Actuator Communication Latency: Random delay $\tau_{\text{lag}} \in [3, 6]\text{ steps}$ ($15\text{ms} \sim 30\text{ms}$)
• Coulomb & Viscous Friction Model:
  • stiff_solref_friction = [-50000.0, -200.0]
  • stiff_solimp_friction = [0.99, 0.9999, 0.001, 0.5, 2.0]

---

3. Massively Parallel RL & Domain Randomization

• Parallel Environments: 4,096 concurrent MuJoCo instances (--env.scene.num-envs 4096)
• Ground Friction Randomization: Uniform sampling $\mu \in [0.4, 1.25]$
• Base Mass & Inertia Jitter: Added payload $\Delta m \in [-0.05, +0.08]\text{ kg}$
• External Force Disturbance (Push): Impulse vector $F_{\text{push}} \in [5.0, 10.0]\text{ N}$ applied every $3 \sim 5\text{ s}$
• Joint Backlash Simulation: Mechanical play $\delta_{\text{backlash}} \in [\pm 0.015\text{ rad}]$

---

4. RSL-RL PPO Training Hyperparameters

• RL Algorithm: Proximal Policy Optimization (PPO) with Actor-Critic MLP
• Control Frequency: 50 Hz ($\Delta t_{\text{ctrl}} = 0.02\text{s}$)
• Physics Solver: MuJoCo 3.12 @ 200 Hz ($dt = 0.005\text{s}$, decimation = 4)
• Actor / Critic Architecture: [512, 256, 128] MLP with ELU activations
• Learning Rate: $1.0 \times 10^{-3}$ (adaptive decay schedule)
• Discount Factor ($\gamma$) / GAE ($\lambda$): $\gamma = 0.99$, $\lambda = 0.95$
• PPO Clip Ratio ($\epsilon$): $\epsilon = 0.2$
• Entropy Regularization: $c_{\text{entropy}} = 0.01$

---

5. Reward Function Design & Weight Matrix

$$\mathcal{R}{\text{total}} = \sum{i} w_i \cdot r_i$$

---

6. 3-Speed Locomotion Benchmarks & Evaluation

<div align="center"> <a href="https://huggingface.co/hwihwalab/microduck-3d-bipedal-teleop/resolve/main/assets/research_benchmark_metrics.png" target="_blank" title="Click to view full resolution benchmark chart"> <img src="assets/research_benchmark_metrics.png" alt="MicroDuck Bipedal Research Benchmark Chart" width="100%" style="border-radius: 8px; border: 1px solid #334460; margin-bottom: 12px; cursor: zoom-in;" /> </a> <p><em>Figure 2: Massively Parallel 4,096 Envs PPO Convergence (196.6M Transitions), 3-Speed Velocity RMSE, Stance Duty Factor, and BAM 7.4V Power Distribution 🔍 Click to view high-res</em></p> </div>

---

🧠 Included 9 Core ONNX Reinforcement Learning Policies

All policies operate on the unified 61D observation vector and output 14D action offsets:

---

📊 Teleoperation Dataset & Imitation Learning Architecture

High-frequency 50Hz teleoperation telemetry generated in the MicroDuck simulation environment follows standard Parquet time-series schemas optimized for Imitation Learning and offline RL:

Telemetry Dataset Features:

• 50Hz Synchronized Time-Series: Full 61D observation vectors, 14D target motor actions, linear/angular velocities, dual-foot contact status, and BAM battery power metrics.
• Physical AI Benchmark Compatibility: Behavioral Cloning (BC), Action Chunking with Transformers (ACT), and Diffusion Policy for Robotics.
• Refer to the Sim-to-Real Research Specifications section above for full tensor and entity breakdown.

---

🕹️ Teleoperation & Unified Keyboard Shortcuts

---

🚀 Quickstart & One-Click Launchers

# 1. Launch Standalone Web 3D Teleoperation Cockpit (Recommended)
.\run_web.ps1
# (or double-click run_web.bat)

# 2. Launch Native C++ MuJoCo Physics Viewer
.\run_simulator.ps1
# (or double-click run_simulator.bat)

# 3. Launch Roller-Skating Simulator
.\run_rollers.ps1
# (or double-click run_rollers.bat)

🌐 Official Project Links

• 🎮 Hugging Face Live Space: https://huggingface.co/spaces/hwihwalab/microduck-3d-bipedal-teleop
• 🤗 Hugging Face Model Hub: https://huggingface.co/hwihwalab/microduck-3d-bipedal-teleop

---

---

🎮 Dual-Mode Locomotion & Autonomous Showcase Pipelines

The system seamlessly switches between Classic Bipedal Walking and High-Speed Roller Skating on the fly with zero physics discontinuity:

stateDiagram-v2
    [*] --> BipedalMode: Default Stand
    BipedalMode --> RollerMode: Press 'M' (Toggle Mode)
    RollerMode --> BipedalMode: Press 'M' (Toggle Mode)
    
    state BipedalMode {
        [*] --> Warmup
        Warmup --> PatrolWander: Balance Locked
        PatrolWander --> GroundPeck: 5.5s
        GroundPeck --> DiscoverBall: 3.5s
        DiscoverBall --> ApproachShoot: Soccer Target
        ApproachShoot --> GoalCelebration: Dynamic Kick
        GoalCelebration --> PoliteRest: 360° Roulade
        PoliteRest --> Standup: Smooth Stand
    }
    
    state RollerMode {
        [*] --> TurboDash
        TurboDash --> AeroGlide: 0.55 m/s (5.0s)
        AeroGlide --> FigureSpin: 2.2s Crouch
        FigureSpin --> RhythmCarving: 360° Pirouette
        RhythmCarving --> CircleDrift: S-Curves
        CircleDrift --> EdgeBrake: Wide Orbit
        EdgeBrake --> TurboDash: Crisp Standstill
    }

1. 🛼 High-Speed Roller-Skating FSM Pipeline (Auto Demo)

• Stage 1 (Turbo Dash): High-speed forward propulsion reaching $\mathbf{0.55\text{ m/s}}$ with soft arena boundary steering.
• Stage 2 (Aero Glide): Aerodynamic low-profile crouch glide ($0.45\text{ m/s}$) utilizing passive rolling wheel momentum.
• Stage 3 (Figure Spin): Continuous graceful 360° pirouette spin around vertical yaw axis ($0.85\text{ rad/s}$).
• Stage 4 (S-Carving Slalom): Sinusoidal edge carving transitions mimicking slalom roller maneuvers.
• Stage 5 (Wide Orbit Drift): High-speed circular sweep power drift towards center arena ($0.38\text{ m/s}$).
• Stage 6 (Edge Brake): Rapid deceleration to crisp standstill finish pose.

2. 🦆 Bipedal Multi-Behavior Soccer & Acrobatics Loop (Auto Demo)

• Wander & Patrol: Smooth sinusoidal wandering stroll with head scan ($0.28\text{ m/s}$).
• Ground Peck: Precision trunk flexion to touch ground with beak and recover posture ($4.0\text{s}$).
• Dynamic Soccer Shoot: Autonomous ball detection, orientation alignment, and alternating single-leg kicking ($1.6\text{s}$).
• Goal Celebration Roulade: Forward 360° tumbling roll acrobatics with automatic self-righting ($2.0\text{s}$).
• Polite Rest: Smooth transition to ground sitting posture and automatic standup.

---

🏟️ 120m Cyber Dual Arena & High-Precision Digital Twin

The Three.js WebGL digital twin cockpit features a high-performance 120-meter Expansive Cyber Arena designed for high-speed skating and bipedal agility testing:

• Dual Mega Grid: Fine sub-grid ($0.5\text{m}$ interval, $240\times 240$ cells) overlaying a major structural grid ($5.0\text{m}$ interval, $24\times 24$ cells).
• Concentric Neon Speed Rings:
  • 1.0m Center Target Ring (Cyan Glow #38bdf8)
  • 3.0m Inner Carving Ring (Teal #06b6d4)
  • 5.0m Rink Perimeter Ring (Sky Blue #0284c7)
  • 8.0m Outer Boundary Speed Ring (Navy #1e3a5f)
• Native C++ Parity: Direct streaming of binary compiled MuJoCo mesh buffers ensuring 100% geometric and kinematic fidelity.

---

📄 License

This project is distributed under the Apache-2.0 License. See the LICENSE file for complete details.

---

Developed and deployed with MicroDuck & MuJoCo 3D Teleoperation by Hwihwa Lab.