lunar-lander-dqn

🚀 LunarLander-v3 Double & Dueling DQN Agent (1,000 Episodes Completed)

A state-of-the-art Deep Q-Network (DQN) trained for 1,000 full episodes on the Gymnasium LunarLander-v3 environment. This agent achieves an exceptional +319.1 Peak Score with flawless horizontal stabilization and perfect dual-leg centered touchdown on the lunar landing pad.

🎯 Model & Algorithm Highlights

• Algorithm: Dueling & Double Deep Q-Network (DDQN)
• Episodes Completed: 1,000 / 1,000
• Architecture:
  • Shared Feature Extraction: Linear(8, 128) -> ReLU -> Linear(128, 128) -> ReLU
  • Value Stream $V(s)$: Linear(128, 64) -> ReLU -> Linear(64, 1)
  • Advantage Stream $A(s, a)$: Linear(128, 64) -> ReLU -> Linear(64, 4)
  • Combined Q-Value: $Q(s, a) = V(s) + \left(A(s, a) - \frac{1}{|A|}\sum_{a'} A(s, a')\right)$
• Stabilization:
  • Double DQN: Decouples action selection (Policy Net) from evaluation (Target Net) to eliminate Q-value overestimation.
  • Polyak Soft Target Updates ($\tau = 0.001$) for seamless convergence.
  • Smooth L1 Loss (Huber Loss) with gradient clipping ($\text{max_norm}=10.0$).
• Exploration Schedule: Epsilon decayed smoothly from $100%$ ($1.0$) down to exactly $5%$ ($0.05$) across $1000$ episodes.

---

📊 Performance & Evaluation

---

💻 How to Use & Evaluate

1. Installation

pip install torch gymnasium "gymnasium[box2d]" huggingface_hub

2. Run Evaluation Script

from huggingface_hub import hf_hub_download
import torch
import gymnasium as gym
from evaluate import DuelingQNetwork

# 1. Download model weights from Hugging Face
model_path = hf_hub_download(repo_id="Taeri077/lunar-lander-dqn", filename="best_model.pth")

# 2. Initialize environment & network
env = gym.make("LunarLander-v3", render_mode="human")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

policy_net = DuelingQNetwork(state_size=8, action_size=4).to(device)
checkpoint = torch.load(model_path, map_location=device)
policy_net.load_state_dict(checkpoint["policy_net"])
policy_net.eval()

# 3. Simulate landing flight
state, info = env.reset(seed=42)
total_reward = 0
done = False

while not done:
    state_tensor = torch.FloatTensor(state).unsqueeze(0).to(device)
    with torch.no_grad():
        action = int(policy_net(state_tensor).argmax(dim=-1).item())
    state, reward, terminated, truncated, info = env.step(action)
    total_reward += reward
    done = terminated or truncated

print(f"🎉 Landing Complete! Total Reward: {total_reward:.2f}")
env.close()

---

🛠️ Final Hyperparameters

{
  "learning_rate": 0.0005,
  "gamma": 0.99,
  "tau": 0.001,
  "batch_size": 64,
  "buffer_capacity": 100000,
  "epsilon_start": 1.0,
  "epsilon_end": 0.05,
  "total_episodes": 1000,
  "loss_function": "SmoothL1Loss"
}

Trained and deployed by Team Leader Jeong & Ddori via Antigravity Lab.