ppo-LunarLander-v3

PPO Agent Playing LunarLander-v2

This is a trained model of a PPO agent playing LunarLander-v2 (the Deep RL Course's Unit 8 environment), implemented from scratch with PyTorch (based on the CleanRL PPO implementation). Trained under gymnasium's current registration id LunarLander-v3 (gymnasium renamed v2 to v3; the underlying environment and physics are unchanged), tagged here as v2 for course/certification consistency.

This repo also contains an earlier Stable-Baselines3 PPO submission for this same environment (ppo-LunarLander-v3.zip / ppo-LunarLander-v3/), kept for Unit 1 of the course; the stable-baselines3 tag above is for that model, not the from-scratch one described here.

Results

Mean reward over 10 evaluation episodes: 279.83 +/- 19.14

Hyperparameters

exp_name: ppo
seed: 1
torch_deterministic: True
cuda: True
track: False
wandb_project_name: cleanRL
wandb_entity: None
capture_video: True
env_id: LunarLander-v3
total_timesteps: 30000000
learning_rate: 0.0003
num_envs: 16
num_steps: 1024
anneal_lr: True
gamma: 0.999
gae_lambda: 0.98
num_minibatches: 32
update_epochs: 4
norm_adv: True
clip_coef: 0.2
clip_vloss: True
ent_coef: 0.01
vf_coef: 0.5
max_grad_norm: 0.5
target_kl: None
repo_id: LATlag/ppo-LunarLander-v3
push_to_hub: True
batch_size: 16384
minibatch_size: 512
num_iterations: 1831