AnuLM-Hindi-QA-400M

Model card: AnuLM-Hindi-QA-400M (ckpt_multi_qa.pt)

A Hindi / English / Python question answerer in the encyclopaedia register: the three-language AnuLM base (AnuLM-Base-400M, step 36,000) fine-tuned on question → lead-paragraph pairs built from Wikipedia and from Python docstrings. Full log and decoded samples: docs/RESULTS.md §23.

Licence: CC BY-SA 4.0. The pairs derive from Hindi and English Wikipedia (CC BY-SA); the base model also saw C4 (ODC-BY) and GitHub Python via codeparrot-clean (mixed licences). Not affiliated with AI4Bharat, BharatGen or the Government of India.

What it does, and does not

Ask X क्या है?, X के बारे में बताइए।, What is X? or What does this function do? and it answers with one or two definition-style sentences. It is not a chatbot: it has never seen a conversation, has no identity, and treats "What is your name?" as a topic to define. It invents details freely; check anything that matters.

Held-out answer loss 3.3150 at the selected step. Manual scoring in docs/RESULTS.md §23: sensible, on-topic answers for well-known subjects in all three languages, degrading quickly on rare ones.

Data

Prompt formats are stored in the checkpoint as qa_templates: प्रश्न: {q} newline उत्तर: for Hindi and Question: {q} newline Answer: for English and Python; loss on the answer tokens only.

Architecture and tokenizer

Same network as every 400M AnuLM checkpoint: 20 layers, hidden 1,024, GQA 16 query / 4 key-value heads, 24 routed experts of 192 with top-4 routing and aux-loss-free balancing, sliding window 256 on layers 0–9, context 512, 32k vocabulary. 397.7M parameters, 173.5M active per token. Tokenizer multi32k (docs/RESULTS.md §22).

Use

python serve.py --ckpt <this folder> then open http://127.0.0.1:8000 and pick answer a question, or python ask.py --ckpt <this folder> on the command line. Temperature 0.3 and repetition penalty 1.3 are the settings the samples were produced with.

How to load

With transformers

The modelling code travels with the weights, so trust_remote_code=True is required and there is nothing to clone:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("toonist/AnuLM-Hindi-QA-400M", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("toonist/AnuLM-Hindi-QA-400M")

ids = tok("\u092a\u094d\u0930\u0936\u094d\u0928: \u0915\u093e\u0936\u0940 \u0915\u094d\u092f\u093e \u0939\u0948?\n\u0909\u0924\u094d\u0924\u0930:", return_tensors="pt")
print(tok.decode(model.generate(**ids, max_new_tokens=60, do_sample=False)[0]))

Greedy output is identical to the AnuLM repository's own generate, cached or not, and the tokenizer here agrees with bpe.py token for token; both are pinned by tests in that repository.

Load it in float32 — which the config now asks for, so the line above is enough. Do not force dtype=torch.bfloat16: this is a mixture-of-experts model whose router keeps a per-expert bias, and rounding that bias to 16 bits changes which experts fire. The output does not get slightly worse, it collapses into repeated tokens. The big tensors are stored in bfloat16 and upcast on load; the router bias and the norms are stored in float32 for this reason. For speed, use torch.autocast over float32 weights, which is how every number in this card was measured. The model loads the code from this repo at trust_remote_code=True; pin a revision if you want that fixed.

Batches must be unpadded (one sequence at a time), and beam search is not supported.

With the AnuLM repository

Every script there takes this folder wherever it takes a .pt:

hf download toonist/AnuLM-Hindi-QA-400M --local-dir AnuLM-Hindi-QA-400M
python serve.py  --ckpt AnuLM-Hindi-QA-400M          # web page at http://127.0.0.1:8000
python sample.py --ckpt AnuLM-Hindi-QA-400M --prompt "\u092a\u094d\u0930\u0936\u094d\u0928: \u0915\u093e\u0936\u0940 \u0915\u094d\u092f\u093e \u0939\u0948?\n\u0909\u0924\u094d\u0924\u0930:"

from model import load_checkpoint, AnuLM
ck = load_checkpoint("AnuLM-Hindi-QA-400M")
m = AnuLM(ck["cfg"]).eval(); m.load_state_dict(ck["model"])

Files: model.safetensors (397.7M parameters), tokenizer.multi32k.json for bpe.BPE.load, and tokenizer.json in the tokenizers format for AutoTokenizer.