lilmainacombat-dqn-policy

Lil Maina Combat - Neural Difficulty Policy

Model: DQN policy that powers the Neural difficulty tier in Lil Maina Combat, a 2D fighting game (Java 26 / Gradle). It learns to fight by self-play against a scripted rule-based opponent and beats every built-in difficulty tier.

Quick facts

State features (normalized)

1. My X / arena width
2. My Y / ground level
3. My health / max health
4. Opponent X / arena width
5. Opponent Y / ground level
6. Opponent health / max health
7. Horizontal distance / arena width
8. Vertical offset / ground level
9. My facing direction
10. Opponent facing direction

Features are mirrored per-agent so the same policy plays both sides of the fight.

Training

• Algorithm: DQN with target network and experience replay (replay buffer capacity 100k).
• Opponent: rule-based controller at the Hard tier, played in a 600-tick time-capped match.
• Reward: health-lead shaping each tick, knockout bonus, anti-turtle penalty so holding at range is never free.
• Exploration: epsilon annealed from 1.0 to 0.05 across a 30k-step final drain; best checkpoint kept by greedy ladder evaluation during training.

Evaluation

Greedy (argmax) policy vs. each built-in rule tier, on two independent 200-game ladder evals (1200 games):

0 losses across all 1200 games. The GUI Neural tier plays this exact greedy policy.

Usage

NeuralNet net = ModelPersistence.loadOrNull("fighter_policy.bin");

The game ships the file as fighter_policy.bin and embeds it in packaged builds (jar / AppImage / Windows .exe) so the Neural tier works self-contained.

Limitations

• Single trained policy; tuned against the rule-based opponent family, not against human playstyles or other learned agents.
• Greedy action selection only (no exploration at inference).
• Q-values are not calibrated probabilities; treat as relative preferences.