Pixelcopter-PLE-v0_

keerthimalladi/Pixelcopter-PLE-v0_

Pixelcopter-PLE-v0_ is a reinforcement learning model from keerthimalladi. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product.

This is a trained model of an Actor-Critic REINFORCE agent playing Pixelcopter-PLE-v0.

Downloads · 30 days

Access

Public

Updated Sep 9, 2026

Repo size

120 KB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.pt40.1 KB · 80%

At a glance

Task: Reinforcement Learning
Access: Public
Created: Sep 9, 2026
Updated: Sep 9, 2026
SHA: 1d4ebc19

Model card

From the Hugging Face model README

Actor-Critic REINFORCE Agent playing Pixelcopter-PLE-v0

This is a trained model of an Actor-Critic REINFORCE agent playing Pixelcopter-PLE-v0.

What changed vs the original (failed) baseline

The default Unit 4 notebook scores around the threshold (mean reward ≲ 5). The model in this repository scores >12 on the same evaluation harness.

Field	Before	After	Why
`h_size`	64	128	Bigger trunk captures non-linear helicopter dynamics
`n_training_episodes`	5 000	30 000	Pixelcopter needs ~20k episodes to converge above the threshold
`gamma`	0.99	0.995	Long-lived episodes — slight horizon extension helps
`lr`	1e-4	5e-4	Faster learning without breaking stability (clipped)
`use_baseline`	false	true	Value baseline reduces policy-gradient variance (the single biggest win)
`normalize_returns`	false	true	Stable advantage estimates across episodes
`entropy_coef`	0	0.01	Keeps exploration alive early — avoids collapsing onto "always thrust"
`grad_clip`	none	1.0	Prevents the occasional catastrophic update

The agent also saves a policy_best.pt whenever the mean evaluation reward improves, so the final checkpoint is always the best one seen during training, not the last one.

To learn to use this model and train yours, check Unit 4 of the Deep Reinforcement Learning Course: https://huggingface.co/deep-rl-course/unit4/introduction

Reproduce locally

pip install torch gym-ple numpy
python train_pixelcopter_improved.py        # ~ 30-60 min on CPU, < 5 min on GPU

Files

train_pixelcopter_improved.py — training script (REINFORCE + value baseline)
hyperparameters.json — all training/eval hyper-parameters
results.json — final evaluation (mean reward)
policy_best.pt — best policy weights (saved when eval improves)
policy_final.pt — last-iteration policy
replay.mp4 — render of the best policy in action

Task: Reinforcement Learning
Created: Sep 9, 2026
Updated: Sep 9, 2026