emergent-snake-locomotion

Emergent Locomotion Patterns of a Snake Robot through Reinforcement Learning

Authors: Yuya Shimizu¹, Yongdong Wang^{1,2}, So Shimooka¹, Tetsushi Kamegawa¹
¹ Graduate School of Environmental, Life, Natural Science and Technology, Okayama University
² Department of Precision Engineering, The University of Tokyo
Project Page: https://yuya-0411.github.io/project-crawl/
Year: 2026

This work was supported by OU-SPRING, JSPS KAKENHI Grant Number 26K07423, and JSPS KAKENHI Grant Number 26K21350.

---

Abstract

Traditional gait design for snake robots has long relied on geometric models inspired by biological snakes. However, due to the inherent morphological discrepancies between biological snakes and robotic systems, these conventional models are not always optimal for snake robots.

In this study, we trained a 10-DoF snake robot on random step fields using reinforcement learning. The learned policy autonomously emerged dynamic gaits capable of traversing unstructured terrains where conventional sine-wave-based locomotion typically fails. The robot effectively utilized environmental protrusions as pivot points for propulsion, three-dimensionally transforming its body to overcome obstacles.

Although these gaits deviate from predefined periodic motions, they exhibit characteristics approximating biological sidewinding as a direct result of physical constraints. Comparative experiments with geometric models confirm that the learned policy offers superior robustness and adaptability in rugged terrains.

---

Robot Model

• Structure: 11 links connected by 10 alternating yaw/pitch joints (orthogonally arranged)
• Length: 9 central units × 0.16 m + 2 terminal units × 0.08 m = 1.6 m total
• Sensors: Each link is equipped with a tactile sensor for contact-aware state feedback

---

Method

Observation & Action Space

The policy takes a 56-dimensional observation as input and outputs 10 joint position targets.

Per-link contact forces are the key embodiment-sensing component — each link's resultant contact force is expressed in its local coordinate frame, capturing normal and tangential friction. Joint targets are scaled as: q_target = (π/2) × a, where a ∈ [−1, 1].

Reward Function

The total reward combines six terms:

Training Setup

Training Terrains (4 types)

---

Checkpoint

model_3000.pt — PPO policy weights at 3,000 training iterations.

import torch

checkpoint = torch.load("model_3000.pt", map_location="cpu")
print(checkpoint.keys())

For full simulation and evaluation setup, see the project page.

---

Key Findings

• The policy emerged a gait approximating biological sidewinding purely from physical constraints — no gait template was provided.
• The robot exploits terrain protrusions as pivot points for propulsion, reconfiguring its body in 3D to overcome obstacles.
• Against the geometric sidewinding baseline (amplitude 1.2 rad, frequency 0.50 Hz), the learned policy shows superior robustness on unstructured terrain, avoiding the "stacking phenomenon" caused by the open-loop geometric model.
• On flat ground, the learned gait achieves reliable forward locomotion driven by the full 56-D observation, with independent per-joint phase control rather than a prescribed waveform.

---

Citation

@inproceedings{shimizu2026emergent,
  title={Emergent Locomotion Patterns of a Snake Robot through Reinforcement Learning},
  author={Shimizu, Yuya and Wang, Yongdong and Shimooka, So and Kamegawa, Tetsushi},
  year={2026}
}