Downloads · 30 days
24
9% of all-time downloads
kaptaan45/KaptaanLM-Vanguard-25M
KaptaanLM-Vanguard-25M is a text generation model from kaptaan45. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
KaptaanLM-Vanguard is the flagship 25.18M parameter model family of the KaptaanLM project. Pre-trained on 5.636 Billion tokens across 8 curated corpora and aligned via the Universal Scratchpad Contract with Schema-Ali…
Downloads · 30 days
24
9% of all-time downloads
All-time downloads
282
Public
Repo size
730 MB
Likes
0
Public
Click a slice to open those files.
.pt730 MB · 100%
From the Hugging Face model README
KaptaanLM-Vanguard is the flagship 25.18M parameter model family of the KaptaanLM project. Pre-trained on 5.636 Billion tokens across 8 curated corpora and aligned via the Universal Scratchpad Contract with Schema-Aligned Direct Preference Optimization (DPO), Vanguard sets a new standard for reasoning density under 50M parameters.
| File Name | Size | Training Tokens | Step Count | Role & Empirical Verdict |
|---|---|---|---|---|
KaptaanLM-Vanguard-Omni-DPO-1.0.pt | 100.7 MB | 50M + DPO | 400 DPO | Overall Grand Champion (CRI: 0.5042, 50% math, 25% logic, 33% code, 3.49% GSM8K, 0.000 rep). |
KaptaanLM-Vanguard-Reasoning-1.0.pt | 100.7 MB | 50.0M (CoT) | 3,000 | Branch B Winner (Base -> CoT, CRI: 0.4167, 66.7% logic, 33.3% math, 33.3% code). |
KaptaanLM-Vanguard-Omni-SFT-1.0.pt | 100.7 MB | 50.0M (Omni) | 3,000 | Omni Joint SFT Baseline (100% scratchpad adherence, CRI: 0.3233). |
KaptaanLM-Vanguard-Instruct-1.0.pt | 100.7 MB | 98.3M + DPO | 400 DPO | Conversational Instruct Winner (Clean dialogue, zero loops). |
KaptaanLM-Vanguard-SFT-1.0-Ablation.pt | 100.7 MB | 98.3M | 6,000 | Instruct SFT Baseline (Exhibited repetition loops on long generations). |
KaptaanLM-Vanguard-1.0.pt | 100.7 MB | 5.636 Billion | 86,000 | Final Base Foundation (Val loss 2.85, 298.4 tokens/non-embed param). |
KaptaanLM-25M-0.1.pt | 125.8 MB | 327.7M - 500M | 5,000 | Pilot Checkpoint (Confirmed GQA 3:1 stability and muP transfer). |
Evaluated under the EleutherAI benchmark protocol under Regime 1 (Deterministic Anti-Repetition Greedy: temp=0.0, rep_penalty=1.12):
| Benchmark Suite | Metric | Questions | Vanguard-Omni-DPO (25M) | Vanguard-Omni-SFT (Non-DPO) | SmolLM-135M Baseline |
|---|---|---|---|---|---|
| ARC-Easy | acc_norm | 2,376 | 32.07% (762) | 29.25% (695) | 44.19% (1,050) |
| ARC-Challenge | acc_norm | 1,172 | 23.81% (279) | 20.99% (246) | 27.82% (326) |
| PIQA | acc_norm | 1,838 | 56.20% (1,033) | 51.25% (942) | 67.46% (1,240) |
| LAMBADA | acc | 5,153 | 13.39% (690) | 0.00% (0) | 24.10% (1,242) |
| GSM8K | acc_norm | 1,319 | 3.49% (46) | 0.08% (1) | 1.52% (20) |
| OVERALL | Total Acc | 11,858 | 23.70% (2,810) | 15.89% (1,884) | 32.70% (3,878) |
import torch
from tokenizers import Tokenizer
from kaptaan.config import CONFIG_25M
from kaptaan.torch.model import KaptaanForCausalLM
tokenizer = Tokenizer.from_file("tokenizer.json")
config = CONFIG_25M
model = KaptaanForCausalLM(config)
model.load_state_dict(torch.load("KaptaanLM-Vanguard-Omni-DPO-1.0.pt", map_location="cpu"))
model.eval()
prompt = "<|im_start|>user\nWrite a Python function to check if a number is prime.\n<|im_end|>\n<|im_start|>assistant\n"
tokens = tokenizer.encode(prompt).ids
input_ids = torch.tensor([tokens], dtype=torch.long)
with torch.no_grad():
output_ids = model.generate(
input_ids,
max_new_tokens=256,
temperature=0.0,
repetition_penalty=1.12,
stop_token_ids=[0, 2]
)
print(tokenizer.decode(output_ids[0].tolist()))
@software{kaptaanlm2026,
author = {Shekhar, Rudransh},
title = {KaptaanLM: Conversational and Reasoning Small Language Models (10M - 100M -> MoE)},
url = {https://github.com/rudy-07/KaptaanLM},
year = {2026}
}