Downloads · 30 days
0
ppo-spaceinvaders-scratch
amaldo123/ppo-spaceinvaders-scratch
ppo-spaceinvaders-scratch is a reinforcement learning model from amaldo123. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product.
This model was trained as part of a project building policy-gradient methods up from first principles: REINFORCE - A2C - A2C+GAE - PPO (this checkpoint).
Access
Public
Updated Sep 21, 2026
Repo size
6.8 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt6.8 MB · 100%
At a glance
- Task
- Reinforcement Learning
- Access
- Public
- Created
- Sep 21, 2026
- Updated
- Sep 21, 2026
- SHA
- 8c362bce
Model card
From the Hugging Face model README
PPO agent for ALE/SpaceInvaders-v5 (from-scratch PyTorch implementation)
This model was trained as part of a project building policy-gradient methods up from first principles: REINFORCE -> A2C -> A2C+GAE -> PPO (this checkpoint).
Environment
ALE/SpaceInvaders-v5, preprocessed with grayscale + 84x84 resize + 4-frame skip + 4-frame stack + terminal-on-life-loss (standard DQN-style preprocessing).
Architecture
Nature CNN trunk (3 conv layers -> 512-d FC) with two heads: policy logits (6 actions) and a scalar state-value.
Training
- Algorithm: PPO, clipped surrogate objective, GAE(lambda=0.95), gamma=0.99
- Rollout length: 128 steps, 4 epochs/rollout, 4 minibatches, clip_eps=0.2
- Optimizer: Adam, lr=2.5e-4
- Total updates: 1000
Evaluation
Mean return over 10 greedy evaluation episodes: 105.0 (std 0.0)
Usage
import torch
from model_def import ActorCritic # see training notebook for the class definition
net = ActorCritic(n_actions=6)
net.load_state_dict(torch.load("ppo_spaceinvaders.pt"))
net.eval()
- Task
- Reinforcement Learning
- Created
- Sep 21, 2026
- Updated
- Sep 21, 2026