ppo-LunarLander-v3

PPO Agent playing LunarLander-v3

This is a trained PPO agent that plays LunarLander-v3, built with the stable-baselines3 library as part of the Hugging Face Deep Reinforcement Learning Course.

The agent achieves a mean reward of 254.57 ± 20.32 over 10 evaluation episodes, which clears the 200 point threshold commonly used as the "solved" criterion for this environment.

The Environment

LunarLander is a classic control task in which the agent must land a spacecraft on a designated pad between two flags. The observation is an 8 dimensional continuous vector containing position, velocity, angle, angular velocity, and left/right leg ground contact flags. The action space is discrete with four options: do nothing, fire the left orientation engine, fire the main engine, or fire the right orientation engine. Firing engines costs fuel (negative reward), crashing is heavily penalized, and a soft landing on the pad is strongly rewarded.

Usage with Stable-Baselines3

import gymnasium as gym
from stable_baselines3 import PPO
from huggingface_sb3 import load_from_hub

repo_id = "NiseRoj/ppo-LunarLander-v3"
filename = "ppo-LunarLander-v3.zip"

# Download the trained model weights from the Hub
checkpoint = load_from_hub(repo_id=repo_id, filename=filename)
model = PPO.load(checkpoint)

# Roll out the policy in a rendered environment
env = gym.make("LunarLander-v3", render_mode="human")
obs, info = env.reset()
for _ in range(1000):
    action, _states = model.predict(obs, deterministic=True)
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        obs, info = env.reset()
env.close()

Evaluating the Agent

import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.evaluation import evaluate_policy
from stable_baselines3.common.monitor import Monitor
from huggingface_sb3 import load_from_hub

checkpoint = load_from_hub(
    repo_id="NiseRoj/ppo-LunarLander-v3",
    filename="ppo-LunarLander-v3.zip",
)
model = PPO.load(checkpoint)

eval_env = Monitor(gym.make("LunarLander-v3"))
mean_reward, std_reward = evaluate_policy(
    model, eval_env, n_eval_episodes=10, deterministic=True
)
print(f"mean_reward={mean_reward:.2f} +/- {std_reward:.2f}")

Training Details

(Replace any of the rows above with the actual values you used if they differ. package_to_hub does not write these for you, so the record is only as accurate as you make it.)

Results

Framework Versions

This model was trained and exported with:

• stable-baselines3
• gymnasium (with the box2d extra for LunarLander)
• huggingface_sb3

See requirements.txt in this repository if you need the exact pinned versions.