holosoma_locomotion

Holosoma FastSAC Locomotion Policy Architecture

Model: fastsac_g1_29dof.onnx
Source: Holosoma Whole-Body Locomotion
Repository: nepyope/holosoma_locomotion
Framework: PyTorch 2.4.1 → ONNX (Opset 13)
Total Parameters: 221,727

---

Table of Contents

1. Overview
2. Input Space (Observation)
3. Output Space (Action)
4. Network Architecture
5. Forward Pass Data Flow
6. PD Gains
7. Comparison with GR00T

---

Overview

The Holosoma FastSAC policy is a single unified actor network trained using the Fast Soft Actor-Critic (SAC) algorithm for bipedal whole-body locomotion. Unlike GR00T's dual-policy system (balance + walk), Holosoma uses one network to handle all locomotion states.

Key Features

• Single frame input: 100D observation (no temporal stacking)
• Full body control: All 29 DOF controlled simultaneously
• Observation normalization: Running mean/std baked into model
• Per-joint action scaling: Action scale factors embedded in ONNX
• LayerNorm: Used instead of BatchNorm for stability
• Gait phase encoding: Sin/cos phase signals for periodic locomotion

---

Input Space (Observation)

Observation Vector (100D)

Total: 100D

Observation Scaling Constants

ANG_VEL_SCALE = 0.25      # Angular velocity scaling
DOF_POS_SCALE = 1.0       # Joint position scaling
DOF_VEL_SCALE = 0.05      # Joint velocity scaling

Input Normalization

The model includes a baked-in observation normalizer applied internally:

# Applied inside ONNX graph
obs_normalized = (obs - obs_mean) / obs_std

# obs_mean: shape (1, 100), range [-0.99, 0.71]
# obs_std:  shape (1, 100), range [0.04, 1.40]

Default Joint Positions (29D Standing Pose)

DEFAULT_ANGLES = np.zeros(29, dtype=np.float32)
DEFAULT_ANGLES[[0, 6]] = -0.312    # hip pitch (left, right)
DEFAULT_ANGLES[[3, 9]] = 0.669     # knee
DEFAULT_ANGLES[[4, 10]] = -0.363   # ankle pitch
DEFAULT_ANGLES[[15, 22]] = 0.2     # shoulder pitch
DEFAULT_ANGLES[16] = 0.2           # left shoulder roll
DEFAULT_ANGLES[23] = -0.2          # right shoulder roll
DEFAULT_ANGLES[[18, 25]] = 0.6     # elbow

Gait Phase

GAIT_PERIOD = 1.0  # seconds
CONTROL_FREQ = 50  # Hz
phase_dt = 2 * π / (CONTROL_FREQ * GAIT_PERIOD)

# Two-phase system (left/right legs offset by π)
phase = [phase_left, phase_right]

# When standing still (no command)
if cmd_magnitude < 0.01:
    phase = [π, π]  # Both legs synchronized (standing)

---

Output Space (Action)

Action Vector (29D)

All 29 joints are controlled:

Total: 29D

Action Scaling & Application

ACTION_SCALE = 0.25  # External scaling applied in control loop

# Target joint positions
target_pos = DEFAULT_ANGLES + action * ACTION_SCALE

Command Ranges

---

Network Architecture

Overview Diagram

Input (100D)
    │
    ▼
┌─────────────────────────┐
│  Observation Normalizer │  (mean/std normalization)
└─────────────────────────┘
    │
    ▼
┌─────────────────────────┐
│  Linear(100 → 512)      │  51,200 params
│  LayerNorm(512)         │  1,024 params
│  ELU()                  │
└─────────────────────────┘
    │
    ▼
┌─────────────────────────┐
│  Linear(512 → 256)      │  131,072 params
│  LayerNorm(256)         │  512 params
│  ELU()                  │
└─────────────────────────┘
    │
    ▼
┌─────────────────────────┐
│  Linear(256 → 128)      │  32,768 params
│  LayerNorm(128)         │  256 params
│  ELU()                  │
└─────────────────────────┘
    │
    ▼
┌─────────────────────────┐
│  Linear(128 → 29)       │  3,712 params
│  Tanh()                 │
└─────────────────────────┘
    │
    ▼
┌─────────────────────────┐
│  Action Scaling         │  action * scale + bias
└─────────────────────────┘
    │
    ▼
Output (29D)

Layer Details

Layer 1: Linear(100 → 512) + LayerNorm + ELU

• Input: 100D normalized observation
• Output: 512D features
• Parameters: 51,200 (weights) + 512 (bias) + 1,024 (LayerNorm) = 52,736

Layer 2: Linear(512 → 256) + LayerNorm + ELU

• Input: 512D features
• Output: 256D features
• Parameters: 131,072 (weights) + 256 (bias) + 512 (LayerNorm) = 131,840

Layer 3: Linear(256 → 128) + LayerNorm + ELU

• Input: 256D features
• Output: 128D features
• Parameters: 32,768 (weights) + 128 (bias) + 256 (LayerNorm) = 33,152

Layer 4: Linear(128 → 29) (Action Head)

• Input: 128D features
• Output: 29D raw actions
• Parameters: 3,712 (weights) + 29 (bias) = 3,741

Weight Summary

Total: 221,727 parameters

---

Forward Pass Data Flow

Complete Forward Pass

def forward(obs_100d):
    """
    Args:
        obs_100d: (batch, 100) observation vector
        
    Returns:
        actions: (batch, 29) joint position deltas
    """
    # 1. Normalize observation (baked into ONNX)
    x = (obs_100d - obs_mean) / obs_std
    
    # 2. Hidden layer 1
    x = linear_0(x)      # (batch, 512)
    x = layer_norm_1(x)
    x = elu(x)
    
    # 3. Hidden layer 2
    x = linear_3(x)      # (batch, 256)
    x = layer_norm_4(x)
    x = elu(x)
    
    # 4. Hidden layer 3
    x = linear_6(x)      # (batch, 128)
    x = layer_norm_7(x)
    x = elu(x)
    
    # 5. Action head
    x = fc_mu(x)         # (batch, 29)
    x = tanh(x)          # Bound to [-1, 1]
    
    # 6. Scale actions (baked into ONNX)
    actions = x * action_scale + action_bias
    
    return actions

Computational Cost

Forward pass operations:
1. Normalization: 100D subtract + divide     (~200 ops)
2. Layer 1: 100→512 matmul + LN + ELU        (~52K MACs)
3. Layer 2: 512→256 matmul + LN + ELU        (~131K MACs)
4. Layer 3: 256→128 matmul + LN + ELU        (~33K MACs)
5. Action head: 128→29 matmul + tanh + scale (~4K MACs)

Total: ~220K multiply-accumulate operations per inference

---

PD Gains

PD gains are stored in ONNX metadata and loaded at runtime.

PD Gain Selection Methodology

Holosoma uses the Raibert-style rule (BeyondMimic method) to compute PD gains from joint armature values:

Kp = I × (2π × fn)²
Kd = 2 × I × ζ × (2π × fn)

Where:

• I = Joint armature (rotational inertia, from URDF/MuJoCo)
• fn = Natural frequency = 10 Hz
• ζ = Damping ratio = 2.0 (over-damped for stability)

This physics-based approach ensures each joint's stiffness/damping is proportional to its inertia.

Verification

Alternative Rule of Thumb (Kyle's Method)

Another common approach in RL locomotion:

Kp = τ_max / RoM   (or τ_max / (RoM/2) for stiffer response)
Kd = Kp / 20

Where τ_max is max torque and RoM is range of motion. This gives similar ballpark values but doesn't account for joint inertia.

Kp (Position Gains)

Kd (Velocity Gains)

References

• Kyle's Rule of Thumb - Kp = τ_max / RoM
• Whole-Body Humanoid Robot Locomotion with Human Reference (Section III.C) - BeyondMimic critical damping method

---

Comparison with GR00T

Key Differences

1. Single vs Dual Policy: Holosoma handles balance and walking with one network; GR00T switches between dedicated balance/walk policies.

2. Temporal Handling: GR00T stacks 6 frames (516D) for temporal context; Holosoma uses gait phase signals (sin/cos) to encode periodicity.

3. Body Coverage: Holosoma controls all 29 DOF including arms; GR00T only controls 15 DOF (legs + waist), leaving arms for other tasks.

4. Architecture: GR00T uses an encoder-decoder (estimator → actor) design with latent normalization; Holosoma is a straightforward MLP with LayerNorm.

---

Usage

from huggingface_hub import hf_hub_download
import onnxruntime as ort
import numpy as np

# Load model
path = hf_hub_download(repo_id="nepyope/holosoma_locomotion", 
                       filename="fastsac_g1_29dof.onnx")
policy = ort.InferenceSession(path)

# Build 100D observation
obs = np.zeros((1, 100), dtype=np.float32)
obs[0, 0:29] = last_action
obs[0, 29:32] = ang_vel * 0.25
obs[0, 32] = cmd_yaw
obs[0, 33:35] = cmd_lin
obs[0, 35:37] = np.cos(phase)
obs[0, 37:66] = (joint_pos - default_pos) * 1.0
obs[0, 66:95] = joint_vel * 0.05
obs[0, 95:98] = gravity
obs[0, 98:100] = np.sin(phase)

# Run inference
action = policy.run(None, {"actor_obs": obs})[0].squeeze()

# Apply to robot
target = default_pos + action * 0.25

---

References

1. Holosoma Project: Whole-body locomotion for humanoid robots
2. SAC Algorithm: "Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning"
3. Unitree G1: https://www.unitree.com/g1

---

Last Updated: 2025-12-17