dqn-SpaceInvadersNoFrameskip-v4

DQN playing SpaceInvadersNoFrameskip-v4

Trained for Kay Zheng (bestdive) as a Hugging Face Deep RL Course project with AI coding and execution assistance. No third-party pretrained weights were used. Free local compute and a free Colab T4 were used.

Training

Stable-Baselines3 DQN, seed 42, 450,000 environment decisions with four-frame skipping and stacking. The first 100,000 decisions were trained locally. The same checkpoint was resumed on Colab for 350,000 more decisions; the replay buffer was restarted with a 10,000-decision warmup. Training uses clipped rewards and episodic life; evaluation uses original full-game rewards with no terminal on life loss. Source scripts and the GPU training log are included.

Independent evaluation

50 full episodes, seeds 100000 through 100049, deterministic actions. Mean reward 392.20, population standard deviation 100.08076738314911, mean minus standard deviation 292.1192326168509. Every episode return is in evaluation.json. Validation used a separate seed range beginning 50000.

Loading

from stable_baselines3 import DQN
from stable_baselines3.common.env_util import make_atari_env
from stable_baselines3.common.vec_env import VecFrameStack
import ale_py

env = VecFrameStack(make_atari_env(
    "SpaceInvadersNoFrameskip-v4", n_envs=1,
    wrapper_kwargs={"clip_reward": False, "terminal_on_life_loss": False}), n_stack=4)
model = DQN.load("model.zip", env=env, device="cpu")
obs = env.reset()
action, _ = model.predict(obs, deterministic=True)

Versions: stable-baselines3 2.9.0, gymnasium 1.3.0, ale-py 0.12.1, torch 2.8.0 on Colab. This is a course model, not a claim of general game mastery.