khazri-2-mini

<p align="center"> <img src="./assets/khazri-2-mini-banner.png" alt="Khazri 2 Mini — compact open-weight language model" width="100%"> </p>

Khazri 2 Mini — 100M

Khazri 2 Mini is a compact, open-weight decoder-only language model in the Khazri family. It combines a modern LLaMA-style Transformer with a custom Byte-Level BPE tokenizer and a 2B-token English training corpus.

Hugging Face · Khazri · Contact

At a glance

Architecture

Training data

Khazri 2 Mini is trained on a custom, pretokenized English-only corpus with a target size of 2,000,000,000 tokens. The corpus is packed into 1,953,125 sequences of 1,024 tokens and stored in Arrow shards with source identifiers.

The documented token budget is:

The listed values are the documented source-token budget. The release manifest should be used for the final source counts of a particular weight revision.

Data processing and safeguards

• Only English text is retained for this corpus. Short records are excluded, very long records are capped, and language/character checks are applied before tokenization.
• The code portion is limited to the eight languages listed above.
• The training mix combines general English, narrative, code, instruction and route-aware synthetic material. This preserves general capabilities while teaching specialized SYNAPSE behaviours.
• Checkpoint evaluation should cover general Q&A, code, arithmetic, context extraction, abstention and current-information requests. A route-specific gain should not be accepted if it degrades general behaviour.
• Original source datasets remain subject to their own terms and licences. Consult their source pages and the model repository licence before use.

Tokenizer

Khazri 2 Mini uses a 32,768-token custom Byte-Level BPE tokenizer. It reserves structural whitespace, chat and SYNAPSE route tokens as single tokens, preserves indentation for code, and uses single-digit splitting to make arithmetic strings more explicit to the model.

Installation

pip install -U torch transformers accelerate safetensors

Quick start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "Yusiko/khazri-2-mini"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype="auto",
    device_map="auto",
)

prompt = "Write a concise explanation of a small language model."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=160,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Khazri 2 Preview comparison

The following table reports the project-provided compact-model comparison for Khazri 2 Preview, not Khazri 2 Mini. Higher is better for every listed task.

These are internal preview results on selected compact-model tests. They are not independently audited and should not be used to make claims about Khazri 2 Mini. Publish prompts, model revisions, scoring rules, hardware and complete evaluation assets with any future benchmark announcement.

Responsible use

Khazri 2 Mini can produce incorrect, incomplete or biased outputs. Evaluate it on your own task, verify material claims and keep a human in the loop for consequential decisions. Do not rely on it as the sole basis for legal, medical, financial, hiring, safety or other high-impact decisions.

Roadmap

Khazri 2 Mini is part of the second Khazri generation. The next planned stage is Khazri 3: a larger parameter scale and stronger results.

Contact

For research, integration or partnership inquiries, visit khazri.dev or email [email protected].