CartPole-kh

ykhalfa/CartPole-kh

CartPole-kh is a reinforcement learning model from ykhalfa. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for stable-baselines3. The card lists the license as mit.

Downloads · 30 days

1% of all-time downloads

All-time downloads

196

Public

Repo size

1.8 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.mp4148 KB · 40%

At a glance

Task: Reinforcement Learning
Library: stable-baselines3
License: mit
Access: Public
Created: Mar 11, 2026
Updated: Mar 22, 2026
SHA: edeab7d1

Model card

From the Hugging Face model README

🤖 CartPole-kh

An A2C agent that perfectly balances a pole — every single time.

</div>

🎬 Demo

The agent keeps the pole perfectly upright for the full 500 steps — the maximum possible score.

🎯 The Task

The CartPole-v1 environment challenges an agent to balance a pole on a moving cart. At each step, the agent pushes the cart left or right to prevent the pole from falling.

	Condition	Value
❌ Fail	Pole angle exceeds	± 12°
❌ Fail	Cart out of bounds	± 2.4 m
✅ Max score	Steps survived	500

Observation space (4 variables):

#	Variable	Description
0	Cart position	Position along the track
1	Cart velocity	Speed of the cart
2	Pole angle	Tilt from vertical
3	Pole angular velocity	Rotation speed

Action space: 0 = push left · 1 = push right

🏗️ Training Details

Parameter	Value
Algorithm	A2C (Advantage Actor-Critic)
Policy	MlpPolicy (actor-critic MLP)
Total timesteps	100 000
Parallel environments	4 (vectorized)
Framework	Stable-Baselines3 + Gymnasium
Experiment tracking	Weights & Biases

Training metrics (reward, policy loss, value loss, entropy) were fully tracked with W&B.

🚀 Quick Start

Install

pip install stable-baselines3 huggingface-sb3 "gymnasium[classic-control]"

Run

from stable_baselines3 import A2C
from huggingface_sb3 import load_from_hub
import gymnasium as gym

# Load from Hub
checkpoint = load_from_hub("ykhalfa/CartPole-kh", "a2c_cartpole.zip")
model = A2C.load(checkpoint)

# Watch it play
env = gym.make("CartPole-v1", render_mode="human")
obs, _ = env.reset()

while True:
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, terminated, truncated, _ = env.step(action)
    if terminated or truncated:
        obs, _ = env.reset()

📊 Evaluation Results

Metric	Value
Mean reward (100 episodes)	500.0 ± 0.0
Success rate	100%

The agent achieves the maximum possible score consistently across all 100 evaluation episodes.

👤 Author

Youssef Khalfa MSO 3.4 — Apprentissage Automatique, École Centrale de Lyon

Task: Reinforcement Learning
Library: stable-baselines3
License: mit
Created: Mar 11, 2026
Updated: Mar 22, 2026