Downloads · 30 days
27
26% of all-time downloads
amadablam-322m
spandyie/amadablam-322m
amadablam-322m is a text generation model from spandyie. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
A 322M-parameter decoder-only LM pretrained from scratch on Nepali, Maithili, and Bhojpuri in three writing systems: Devanagari, IAST, and natural phonetic romanization. 12 of 14 layers use linear attention with per-h…
All-time downloads
102
Public
Parameters
322M
1.3 GB on disk
Likes
1
Trending 1
Repo makeup
Click a slice to open those files.
.safetensors1.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- amadablam
- Access
- Public
- Created
- Jul 19, 2026
- Updated
- Jul 19, 2026
- SHA
- 034bb556
Model card
From the Hugging Face model README
Ama Dablam 322M (base)
A 322M-parameter decoder-only LM pretrained from scratch on Nepali, Maithili, and Bhojpuri in three writing systems: Devanagari, IAST, and natural phonetic romanization. 12 of 14 layers use linear attention with per-head exponential decay (base 0.7); 2 use full softmax. 17.1B training tokens, 16,384-entry trilingual BPE.
| val bpb (lower=better) | ne | mai | bho | iast_ne | phon_ne |
|---|---|---|---|---|---|
| Ama Dablam base | 0.406 | 0.368 | 0.546 | 0.326 | 0.942 |
| Manaslu 491M dense (prior) | 0.432 | 0.438 | 0.632 | 0.386 | — |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("spandyie/amadablam-322m")
model = AutoModelForCausalLM.from_pretrained("spandyie/amadablam-322m",
trust_remote_code=True)
ids = tok("नेपालको इतिहास", return_tensors="pt").input_ids
print(tok.decode(model.generate(ids, max_new_tokens=100, do_sample=True)[0]))
Details & limitations
- Config serializes
linear_decay=0.7— required at construction; handled automatically here. - Base model: no instruction following (see
spandyie/amadablam-322m-instruct). - This model is cacheless by design.
generation_config.jsonsetsuse_cache: falseby default. If you override generation kwargs, do not setuse_cache=True— it will silently produce incoherent, context-free output rather than an error (only the last token gets fed back each step). - Right-pad with the BOS token (id 16368); attention_mask is accepted but unused.
- 322M params / 17.1B tokens: factual knowledge is limited; single-seed training run.
- Pretraining corpus: web/news/wiki + curated Nepali sets, dedup + quality-gated; phonetic slices generated by deterministic transliteration.
- Task
- Text Generation
- Library
- transformers
- Type
- amadablam
- License
- apache-2.0
- Languages
- ne, mai, bho
- Created
- Jul 19, 2026
- Updated
- Jul 19, 2026