QaptaanLM-0.75B-Instruct

QaptaanLM-0.75B-Instruct: Compact Hybrid-Attention AI Programming Assistant

QaptaanLM-0.75B-Instruct is the instruction-aligned programming and technical reasoning assistant built on QaptaanLM-0.75B. Fine-tuned on KapInstruct-100M using Qwen ChatML formatting and assistant-only loss masking, the model excels at multi-language code generation, algorithmic reasoning, debugging, and constraint-based instruction following.

---

🌐 Model Ecosystem & Deployment Formats

---

Model Specification

---

Recommended Generation Parameters (SFT Instruct)

generation_config = {
    "do_sample": True,
    "temperature": 0.20,              # Optimal balance of determinism & creative reasoning
    "top_p": 0.90,                    # Nucleus sampling threshold
    "top_k": 40,                      # Restricts to top-40 candidate tokens
    "repetition_penalty": 1.12,       # Prevents repetitive code generation loops
    "eos_token_id": [248044, 248046], # <|endoftext|> (248044) and <|im_end|> (248046)
    "pad_token_id": 248044
}

---

Quickstart & Usage

1. ChatML Dialogue Inference via transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "kaptaan45/QaptaanLM-0.75B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "system", "content": "You are QaptaanLM, an expert AI programming assistant."},
    {"role": "user", "content": "Write a Python function `def is_palindrome(s: str) -> bool:` with docstring and examples."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        temperature=0.20,
        top_p=0.90,
        repetition_penalty=1.12,
        eos_token_id=[248044, 248046],
    )

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

---

Dataset Attribution

Fine-tuned on KapInstruct-100M across 12 diverse instruction domains:

• Code Generation & Programming [31%]: Magicoder-Evol (13%), Magicoder-OSS (8%), Self-OSS-Instruct (5%), Smol-Constraints (3%).
• General Dialogue & Reasoning [27%]: Smol-Magpie-Ultra (18%), OpenHermes-2.5 (9%).
• Mathematical Reasoning (CoT) [17%]: OpenMathInstruct-2 (11%), NuminaMath-CoT (6%).
• Debugging & Error Repair [10%]: CodeFeedback-Filtered (10%).
• STEM & Scientific QA [11%]: OpenThoughts-114k (7%), WebInstructSub (4%).
• Strict Constraint Adherence [4%]: Tulu-3-SFT (6%), Smol-Constraints (3%).

---

License

Released under the Apache 2.0 License.