KaptaanLM-Scout-10M

KaptaanLM-Scout-10M: Master Fleet Repository

KaptaanLM-Scout is a family of ultra-compact autoregressive small language models (9.95M parameters) built from scratch for natural multi-turn conversation and multi-step reasoning on edge and resource-constrained devices.

• GitHub Repository: https://github.com/rudy-07/KaptaanLM
• Evaluation Protocol: EleutherAI lm-evaluation-harness (11,858 official academic questions)
• Architecture: 8 layers, d_model=256, 8 attention heads, 4 KV heads (2:1 GQA), d_ffn=680 (SwiGLU), 16,384 Tied BPE vocabulary
• Context Length: 2,048 tokens (512 sliding window attention)
• Total Parameters: 9,949,952 (9.95M)
• Non-Embedding Parameters: 5,755,648 (5.76M / 57.85% reasoning capacity)

---

Model Fleet Directory (Checkpoints Available in this Repository)

---

Official Academic Benchmark Results (11,858 Questions)

Evaluated under the EleutherAI benchmark protocol under Regime 1 (Deterministic Anti-Repetition Greedy: temp=0.0, rep_penalty=1.12):

Parameter Efficiency Advantage

• GSM8K Mathematical Reasoning Density: Scout-Omni achieves 0.2593% accuracy per 1M parameters, representing a 22.95x parameter efficiency advantage over SmolLM-135M (and 31.32x non-embedding efficiency).
• Overall Suite Density: Scout-Omni delivers 2.0816% accuracy per 1M parameters (8.56x SmolLM efficiency).

---

Usage & Python Quickstart

import torch
from tokenizers import Tokenizer
from kaptaan.config import CONFIG_10M
from kaptaan.torch.model import KaptaanForCausalLM

tokenizer = Tokenizer.from_file("tokenizer.json")
config = CONFIG_10M
model = KaptaanForCausalLM(config)
model.load_state_dict(torch.load("KaptaanLM-Scout-Omni-SFT-1.0.pt", map_location="cpu"))
model.eval()

prompt = "<|im_start|>user\nSolve: A train travels 120 miles in 2 hours. At this speed, how far does it travel in 5 hours?<|im_end|>\n<|im_start|>assistant\n"
tokens = tokenizer.encode(prompt).ids
input_ids = torch.tensor([tokens], dtype=torch.long)

with torch.no_grad():
    output_ids = model.generate(
        input_ids,
        max_new_tokens=256,
        temperature=0.0,
        repetition_penalty=1.12,
        stop_token_ids=[0, 2]
    )

print(tokenizer.decode(output_ids[0].tolist()))

Citation

@software{kaptaanlm2026,
  author = {Shekhar, Rudransh},
  title = {KaptaanLM: Conversational and Reasoning Small Language Models (10M - 100M -> MoE)},
  url = {https://github.com/rudy-07/KaptaanLM},
  year = {2026}
}