lunarlander-v3-d3qn

🚀 LunarLander-v3 D3QN // Mission Control Deck

“LunarLander-v3 D3QN // Mission Control: Autonomous Lunar Soft Landing & Aerospace Telemetry HUD | HWIHWA LAB
High-precision attitude stabilization, vertical velocity damping, and soft touchdown between lunar landing flags using Dueling Double Deep Q-Network (D3QN) and cold-gas RCS thruster physics.

🏛️ Foundational Research Baseline: This project establishes the core algorithmic and mathematical foundation for Hwihwa Lab's advanced Tesla Roadster 180° Inversion Flip-Landing Study (D3QN & PPO).

🌐 Live Interactive Space | 📦 English Model Card | 🇰🇷 한국어 매뉴얼 | 🏎️ Roadster D3QN | 🎯 Roadster PPO”
Developed by Hwihwa Lab · AI × Automation × Aesthetics ✨

LunarLander D3QN Mission Control Deck is an advanced Dueling Double Deep Q-Network (D3QN) reinforcement learning system paired with a zero-scroll, real-time 3-column aerospace telemetry dashboard engineered for high-precision, stylish lunar touchdowns.

---

📊 Model Specifications & Benchmark Results

---

🌟 Core Algorithmic Highlights

1. 🧠 Dueling Double DQN (D3QN) Architecture

• Dueling Stream Decomposition: Decomposes Q-values into state-value $V(s)$ and action advantage $A(s, a)$ to accelerate policy convergence on non-action-critical states.
• Double DQN Value Estimation: Decouples action selection (Online Network) from value estimation (Target Network) to eliminate severe Q-value overestimation bias.
• Robust Training Optimization: Huber Loss (Smooth L1), AdamW optimizer, LayerNorm feature normalization, and gradient clipping.

2. ✨ Stylish Landing Custom Reward Shaping (shape_stylish_reward)

• Attitude & Angular Lock: Enforces horizontal hull orientation ($|\theta| < 0.08$) and penalizes excessive rotational spin ($|\omega| < 0.1$).
• Soft Touchdown Velocity Damping: Rewards smooth deceleration upon ground proximity ($-0.3 \le V_y \le 0.0$).
• Precision Center-Pad Bonus: Substantial reward incentive for simultaneous dual-leg ground contact directly between landing flags.

---

🖥️ 3-Column Mission Control Deck (One-Screen Fitted Design)

Engineered as a 1536x762 zero-scroll, high-density aerospace flight telemetry dashboard (overflow: hidden; height: 100vh;).

+---------------------------------------------------------------------------------------------+
| 🚀 HEADER BAR : Brand | EPISODE (0/1000) | EXPLORATION (ε: 100%→5%) | BEST SCORE (+311.16)  |
+------------------------------+-------------------------------+------------------------------+
| 👈 LEFT: FLIGHT HUD          | 🛸 CENTER: SIMULATION STREAM  | 👉 RIGHT: TRAINING ANALYTICS |
| - Attitude & Pitch Gyro      | - High-FPS Apollo 2D Canvas   | - 4 Stats (Score, SMA, etc.) |
| - Velocity Gauges (Vy, Vx)   | - Mission Control Action Deck | - Chart 1: Reward & 100-MA   |
| - Coordinates & Leg Sensors  | - Autopilot / Manual / Train  | - Chart 2: Epsilon Decay     |
| - 4-Action DQN Q-Values      | - Time Warp Speed Multipliers | - Chart 3: MSE Loss Progress |
| - Mission Avionics Specs     | - Strict Quality Tiers Banner | - D3QN Architecture Specs    |
+------------------------------+-------------------------------+------------------------------+

1) 👈 Left Column: FLIGHT TELEMETRY HUD

• Artificial Horizon (Pitch Gyro): Dynamic attitude indicator rotating with vessel pitch angle ($\theta$) and angular velocity ($\omega$).
• Velocity Vector Bars: Real-time vertical ($V_y$) and horizontal ($V_x$) velocity progress meters with caution/danger color thresholds.
• Coordinates & Contact Sensors: Real-time altitude ($Y$), pad offset ($X$), and left/right landing gear contact indicators.
• 4-Action DQN Q-Values: Live Q-value bars for 0: IDLE (COAST), 1: LEFT THRUSTER, 2: MAIN THRUSTER, 3: RIGHT THRUSTER.
• Mission Avionics & Sensors: Real-time hardware telemetry specs card (Inertial Measurement Unit, LIDAR altimeter, RCS solenoid valves).

2) 🛸 Center Column: SIMULATION STREAM & FLIGHT CONTROLS

• Hardware-Accelerated 2D Canvas: Apollo/SpaceX lander sprite, plasma exhaust particle engines, neon trajectory motion trails, Box2D terrain with landing flags ($|X| \le 0.20$), and golden touchdown fireworks.
• Integrated Mission Controls:
  • [▶ START MISSION]: Immediately initiates continuous autonomous flights (in Autopilot mode) or active flight control.
  • [⏸ PAUSE], [🔄 RESET], [📥 LOAD BEST MODEL].
• 3 Unified Operating Modes:
  • 🤖 AUTOPILOT (DEMO) (Default): Continuous autonomous landing showcase using the pre-trained D3QN checkpoint ($\epsilon=0.0$) across curated mission seeds with 100% touchdown success and a 1.2s celebration pause between flights.
  • 🎮 MANUAL FLIGHT: Interactive human keyboard flight simulator (W/A/D, Arrow keys, or Spacebar).
  • ⚡ LIVE TRAINING: Live reinforcement learning with dynamic epsilon decay ($1.0 \to 0.05$) and experience replay updates.
• Speed Multipliers: 1x, 2x, 5x, ⚡ MAX.
• Strict Touchdown Quality Validation (No False Positives):
  • PERFECT SOFT TOUCHDOWN! 🏆: Dead-center ($|X| \le 0.08$), ultra-soft descent ($|V_y| \le 0.35$), dual-leg contact.
  • TOUCHDOWN SUCCESS! 🎉: Strictly on-pad between flags ($|X| \le 0.20$), safe descent ($|V_y| \le 0.55$), leg contact.
  • OFF-PAD LANDING (OUT OF BOUNDS) ⚠️: Safe upright resting, but outside the designated landing pad flags.
  • CRASH DETECTED 💥: Excessive touchdown impact, capsized orientation, or hull collision.

3) 👉 Right Column: DQN TRAINING ANALYTICS

• 4 Real-time Metrics: CURRENT SCORE, SMA 100 AVG, SUCCESS RATE (%), TRAINING LOSS.
• 3-Tier Synchronized Multi-Charts (Chart.js):
  1. EPISODE REWARD & 100-MA (Solved Threshold: > 200.0)
  2. EPSILON (100% → 5% DECAY) (Exploration decay curve)
  3. MSE LOSS PROGRESSION (Neural network loss trajectory)
• D3QN Architecture Specs: Detailed neural network topology, Soft Polyak update, and AdamW hyperparameter specs.

---

🎮 Manual Flight Controls (🎮 MANUAL FLIGHT Mode)

---

🚀 Quick Start Guide

1. Launch Mission Control Dashboard (Recommended)

python run.py

Open your browser and navigate to http://localhost:8000 to access the real-time aerospace control deck.

2. Standalone CLI High-Speed Training

python dqn_lunalander.py --cli-train

3. Deploy to Hugging Face Model Hub

python deploy_to_hf.py

---

💻 Quick Evaluation Snippet

You can load and evaluate this pre-trained agent in 5 lines of Python:

import torch, gymnasium as gym
from dqn_agent import DQNAgent

# 1. Initialize environment & agent
env = gym.make("LunarLander-v3", render_mode="human")
agent = DQNAgent(state_size=8, action_size=4)
agent.load("models/best_model.pth")

# 2. Run greedy landing evaluation
state, _ = env.reset()
done = False
while not done:
    action, _ = agent.act(state, eps=0.0)
    state, reward, terminated, truncated, _ = env.step(action)
    done = terminated or truncated

env.close()

---

📂 Repository Contents

• best_model.pth: Pretrained PyTorch Dueling Double DQN neural network weights (+311.16 score).
• config.json: Model architecture, state/action specifications, and hyperparameter configuration.
• dqn_agent.py: Standalone PyTorch implementation of DQNAgent and DuelingQNetwork.
• README.md: Official global English model card and benchmark documentation.
• README_KR.md: Comprehensive Korean technical manual and aerospace telemetry specifications.
• LICENSE: MIT License.

---

👤 About Hwihwa Lab

“AI × Automation × Aesthetics — Crafting tools you'll love ✨”
Hwihwa Lab is an independent AI research & engineering studio dedicated to blending practical on-device intelligence, autonomous operations, and refined aesthetics. We engineer resilient systems that free solopreneurs and indie creators from operational overhead—giving people their time and creative joy back.

🔬 Core Research & Engineering Pillars

• Zero-Cost Edge Operations: Private, on-device AI pipelines for autonomous business workflows without ongoing cloud API overhead.
• Autonomous Decision OS & Micro-ERP: Deterministic, prefrontal-cortex-style reasoning architectures designed to orchestrate daily business decisions and operations.
• Embodied AI & Domain-Specific Robotics: Simulation-driven reinforcement learning, physical control brains, and specialized domain data engineering for real-world service environments.
• Interactive Creative Aesthetics: Fusing functional automation with refined visual craft and modern design philosophy to build delightful tools.

🌐 Connect & Aerospace RL Ecosystem

• Research Studio: 🤗 Hugging Face (@hwihwalab) · 🐙 GitHub (@Hwihwa-Lab)
• 🚀 Autonomous Flight & Landing Control Lineup:
  • 🌕 Phase 1 · Classical Soft Landing (Current Model / Foundational Baseline): lunarlander-v3-d3qn Model Hub
  • 🛸 Phase 1 · Live Interactive Mission Control Deck: lunarlander-v3-d3qn Spaces (Zero-server 60fps in-browser flight deck)
  • 🏎️ Phase 2 · Acrobatic Inversion Recovery (D3QN): tesla-roadster-flip-landing-d3qn (100% touchdown rate with cautious attitude control)
  • 🎯 Phase 2 · Acrobatic Inversion Recovery (PPO): tesla-roadster-flip-landing-ppo (98% touchdown rate with ultra-fast agile ballistic descent)
  • 🛸 Dual-Engine Interactive Cockpit: tesla-roadster-flip-landing (Real-time in-browser 60fps comparison deck)

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

Trained and deployed with LunarLander Mission Control by HWIHWALAB.