custom-dqn-space-invaders

Deep Q-Network (DQN) from Scratch - SpaceInvadersNoFrameskip-v4

This repository contains a custom Convolutional Neural Network (CNN) trained using Deep Q-Learning completely implemented from scratch in pure PyTorch.

Training Configuration & Metrics

• Environment: SpaceInvadersNoFrameskip-v4 (Atari)
• Total Timesteps: 1,000,000
• Final Evaluation Scores: ~400+ points
• Optimizer: Adam (Learning Rate: 1e-4)
• Framework: Pure PyTorch & Gymnasium (AtariWrapper)

Model Architecture

class AtariDQN(nn.Module):
    def __init__(self, n_actions):
        super(AtariDQN, self).__init__()
        
        # 1. Feature Extractor (Scans 4 stacked gray image channels)
        self.cnn = nn.Sequential(
            nn.Conv2d(4, 32, kernel_size=8, stride=4),
            nn.ReLU(),
            nn.Conv2d(32, 64, kernel_size=4, stride=2),
            nn.ReLU(),
            nn.Conv2d(64, 64, kernel_size=3, stride=1),
            nn.ReLU(),
            nn.Flatten()
        )
        
        # 2. Fully Connected Classifier Layers (Predicts action scores)
        self.fc = nn.Sequential(
            nn.Linear(64 * 7 * 7, 512),
            nn.ReLU(),
            nn.Linear(512, n_actions) # <--- Cleaned up to match 'n_actions' perfectly
        )
        
    def forward(self, x):
        # Scale pixels from [0, 255] down to float standard [0.0, 1.0]
        return self.fc(self.cnn(x / 255.0))