claude-player
An AI-powered Game Boy emulator agent that uses Claude's vision and reasoning to autonomously play Game Boy games.
SKILL.md
Full skill instructions
ClaudePlayer
An AI agent that plays Game Boy games autonomously using Claude's vision capabilities and the PyBoy emulator.
Overview
ClaudePlayer connects Claude to a Game Boy emulator. Each turn, Claude receives a screenshot of the current game frame, reasons about what to do, and sends button inputs back to the emulator. It maintains a structured memory system to track game progress (items, NPCs, locations, quests, stats).
Prerequisites
- Python 3.10+
- Pipenv
- An Anthropic API key
- A Game Boy ROM file (
.gb)
Setup
# Install dependencies
pipenv install
# Create .env with your API key
echo "ANTHROPIC_API_KEY=your_key_here" > .env
# Place a Game Boy ROM in the project directory
Configuration
Edit config.json:
| Key | Description | Default |
|---|---|---|
ROM_PATH | Path to Game Boy ROM | Required |
STATE_PATH | Saved emulator state file | null |
EMULATION_MODE | "turn_based" or "continuous" | "turn_based" |
MODEL_DEFAULTS.MODEL | Claude model to use | "claude-sonnet-4-5-20250929" |
MODEL_DEFAULTS.THINKING | Enable extended thinking | true |
RATE_LIMITS.RPM_THRESHOLD | Requests per minute limit | Configured in file |
RATE_LIMITS.TPM_THRESHOLD | Tokens per minute limit | Configured in file |
REDIS_LOGS | Optional Redis logging config | null |
Running
pipenv shell
# Default config
python play.py
# Custom config
python play.py --config my_config.json
# Create a saved emulator state
python emu_setup.py
Project Structure
| Path | Purpose |
|---|---|
play.py | Launcher script |
claude_player/main.py | CLI entry point with arg parsing |
claude_player/agent/game_agent.py | Main orchestrator: emulator init, game loop, coordination |
claude_player/interface/claude_interface.py | Claude API communication and rate limiting |
claude_player/state/game_state.py | Game state tracking (memory, goals, history) |
claude_player/tools/tool_setup.py | Tool definitions (send_inputs, memory ops, etc.) |
claude_player/tools/tool_registry.py | Tool registry system |
claude_player/config/config_loader.py | Config file parsing |
claude_player/config/config_class.py | Configuration data class |
claude_player/utils/game_utils.py | Button input parsing, screenshot capture |
claude_player/utils/memory_reader.py | Game memory/stats reader |
claude_player/agent/summary_generator.py | Periodic game progress summarization |
config.json | Runtime configuration |
Available Tools (for the agent during gameplay)
| Tool | Description |
|---|---|
send_inputs | Send button sequences to the emulator |
set_game | Identify the current game |
set_current_goal | Update the gameplay objective |
add_to_memory | Store items, NPCs, locations, quests, mechanics, stats |
remove_from_memory | Remove a memory entry |
update_memory_item | Update an existing memory entry |
toggle_thinking | Toggle extended thinking mode on/off |
How It Works
- PyBoy emulator loads the ROM and renders a frame
- The frame is captured as a screenshot and sent to Claude
- Claude analyzes the screen, reasons about the game state, and calls tools
send_inputspresses buttons on the emulator (A, B, Up, Down, Left, Right, Start, Select)- In turn-based mode, the emulator only advances when the agent acts
- The agent maintains structured memory to track long-term game progress
- Periodic summaries compress the conversation history to stay within context limits