ppo-LunarLander-v2

PPO Agent — LunarLander-v2

Trained PPO agent on LunarLander-v2 (Gymnasium) as part of the Hugging Face Deep RL Course — Unit 1.

Evaluation Results

Evaluation used VecNormalize-wrapped env with normalization stats loaded from vec_normalize.pkl, deterministic=True.

Environment

Training Hyperparameters

EvalCallback: eval every 10,000 steps, 10 eval episodes, saves best model checkpoint.

Usage

from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv, VecNormalize
from stable_baselines3.common.monitor import Monitor
from huggingface_hub import hf_hub_download
import gymnasium as gym

# Load model
model = PPO.load(hf_hub_download("muhrivandysetiawan/ppo-LunarLander-v2", "PPO-LunarLander-v2.zip"))

# Load normalization stats
vec_path = hf_hub_download("muhrivandysetiawan/ppo-LunarLander-v2", "vec_normalize.pkl")
eval_env = DummyVecEnv([lambda: Monitor(gym.make("LunarLander-v2", render_mode="rgb_array"))])
eval_env = VecNormalize.load(vec_path, eval_env)
eval_env.training = False
eval_env.norm_reward = False

obs = eval_env.reset()
done = False
while not done:
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, done, info = eval_env.step(action)

Notes

• No W&B integration in this version — TensorBoard logging used during training (./ppo_lunarlander_tensorboard/).
• vec_normalize.pkl must be loaded alongside the model for correct inference; omitting it degrades performance significantly.
• Trained on Google Colab with virtual display (pyvirtualdisplay).