lunarlander-v3-dqn-scratch

Deep Q-Network (DQN) from Scratch - LunarLander-v3

This repository contains a PyTorch implementation of a Deep Q-Network (DQN) built from scratch to solve the LunarLander-v3 environment from Gymnasium.

Agent Demonstration

https://huggingface.co/arabellako22/lunarlander-v3-dqn-scratch/resolve/main/replay.mp4

<video controls autoplay loop> <source src="https://huggingface.co/arabellako22/lunarlander-v3-dqn-scratch/resolve/main/replay.mp4" type="video/mp4"> </video>

Architecture & Implementation Details

• Neural Network: 3-Layer Fully Connected Network (8 -> 64 -> 64 -> 4)
• Experience Replay: Uniform Experience Replay Buffer (Capacity: 100,000)
• Target Network: Soft target updates with $\tau = 0.001$
• Loss Function: Mean Squared Error (MSE) with Adam Optimizer (LR = 0.0005)
• Exploration: $\epsilon$-greedy strategy with decay from 1.0 to 0.01

How to Run Inference

import torch
import gymnasium as gym
from model import QNetwork

# Initialize environment and load weights
env = gym.make("LunarLander-v3", render_mode="human")
model = QNetwork(state_size=8, action_size=4)
model.load_state_dict(torch.load("model.pt", map_location="cpu"))
model.eval()

state, _ = env.reset()
done = False

while not done:
    state_tensor = torch.from_numpy(state).float().unsqueeze(0)
    with torch.no_grad():
        action = torch.argmax(model(state_tensor)).item()
    state, reward, terminated, truncated, _ = env.step(action)
    done = terminated or truncated

env.close()