video-game-identification-sleuth

Sleuth - Video Game Screenshot Classifier (v1)

Sleuth is an EfficientNet-B0-based image classifier that identifies which video game a gameplay screenshot is from. This is the first iteration of the model.

Model details

• Architecture: EfficientNet-B0 (transfer learning from ImageNet weights)
• Input: RGB image, any resolution (resized + padded to 224x224 at inference)
• Output: Probability distribution over 20 game classes
• Test accuracy: ~97% overall on the held-out test split

Supported games (20)

• DOTA 2
• Counter-Strike 2
• Terraria
• Rust
• Dead by Daylight
• Resident Evil 7 Biohazard
• PUBG: BATTLEGROUNDS
• Resident Evil 2 Remake
• Resident Evil 3 Remake
• Resident Evil Village
• Fears to Fathom - Home Alone (Episode 1)
• Fears to Fathom - Norwood Hitchhiker (Episode 2)
• ARC Raiders
• Resident Evil 4 Remake
• Fears to Fathom - Carson House (Episode 3)
• Fears to Fathom - Ironbark Lookout (Episode 4)
• Delta Force
• Fears to Fathom - Woodbury Getaway (Episode 5)
• Grand Theft Auto V
• Resident Evil Requiem

Known limitations

• Accuracy (~97%) was measured on clean, unmodified gameplay screenshots from the training distribution. In real-time / real-world usage, accuracy can drop noticeably when the input frame has:
  • Streaming overlays (chat, webcam, alerts, donation goals, etc.)
  • Color filters, LUTs, or heavy post-processing
  • Aspect ratios or UI scaling very different from the training data
  • Heavy compression artifacts (e.g. low-bitrate stream captures)
• The model has only been trained on the 20 games listed above - screenshots from other games will be forced into one of these classes (no "unknown" class).
• Visually similar titles (e.g. multiple Resident Evil entries) are more prone to confusion than visually distinct games.

Usage

import torch
from torchvision.transforms import Compose, ToTensor, Normalize, Pad
from efficientnet_pytorch import EfficientNet
from PIL import Image

class ResizeAndPad:
    def __init__(self, target_size=224):
        self.target_size = target_size

    def __call__(self, img):
        w, h = img.size
        if w > h:
            new_w = self.target_size
            new_h = int(h * self.target_size / w)
        else:
            new_h = self.target_size
            new_w = int(w * self.target_size / h)
        img = img.resize((new_w, new_h), Image.BILINEAR)
        pad_w = self.target_size - new_w
        pad_h = self.target_size - new_h
        padding = (pad_w // 2, pad_h // 2, pad_w - pad_w // 2, pad_h - pad_h // 2)
        return Pad(padding, fill=0)(img)

transform = Compose([
    ResizeAndPad(224),
    ToTensor(),
    Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

mapping_steamdbid_game = {
    570: 'DOTA 2',
    730: 'Counter-Strike 2',
    105600: 'Terraria',
    252490: 'Rust',
    381210: 'Dead by Daylight',
    418370: 'Resident Evil 7 Biohazard',
    578080: 'PUBG: BATTLEGROUNDS',
    883710: 'Resident Evil 2 Remake',
    952060: 'Resident Evil 3 Remake',
    1196590: 'Resident Evil Village',
    1671340: 'Fears to Fathom - Home Alone (Episode 1)',
    1763050: 'Fears to Fathom - Norwood Hitchhiker (Episode 2)',
    1808500: 'ARC Raiders',
    2050650: 'Resident Evil 4 Remake',
    2120900: 'Fears to Fathom - Carson House (Episode 3)',
    2506160: 'Fears to Fathom - Ironbark Lookout (Episode 4)',
    2507950: 'Delta Force',
    2961530: 'Fears to Fathom - Woodbury Getaway (Episode 5)',
    3240220: 'Grand Theft Auto V',
    3764200: 'Resident Evil Requiem',
}

checkpoint = torch.load("game_classifier_best_new.pth", map_location="cpu")
steamdb_to_idx = {steamdb_id: idx for idx, steamdb_id in enumerate(mapping_steamdbid_game.keys())}
idx_to_steamdb = {idx: steamdb_id for steamdb_id, idx in steamdb_to_idx.items()}

model = EfficientNet.from_name("efficientnet-b0", num_classes=checkpoint["num_classes"])
model.load_state_dict(checkpoint["model_state_dict"])
model.eval()

image = Image.open("screenshot_videogame.png").convert("RGB")
input_tensor = transform(image).unsqueeze(0)

with torch.no_grad():
    outputs = model(input_tensor)
    probabilities = torch.softmax(outputs, dim=1)[0]
    top_prob, top_idx = probabilities.max(0)

steamdb_id = idx_to_steamdb[top_idx.item()]
print(f"{mapping_steamdbid_game[steamdb_id]}: {top_prob.item() * 100:.2f}%")