Downloads · 30 days
19
46% of all-time downloads
AsifZaman1912/ChatgaiyyaLM-0.5B-B2_subset_ctg_syl_bn
ChatgaiyyaLM-0.5B-B2_subset_ctg_syl_bn is a machine learning model from AsifZaman1912. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter adapting Qwen/Qwen2.5-0.5B to Chittagonian (চাটগাঁইয়া), a Bangla dialect with tens of millions of speakers and almost no NLP resources.
Downloads · 30 days
19
46% of all-time downloads
All-time downloads
41
Public
Repo size
3.4 GB
Likes
0
Public
Click a slice to open those files.
.safetensors3.4 GB · 100%
From the Hugging Face model README
LoRA adapter adapting Qwen/Qwen2.5-0.5B to Chittagonian (চাটগাঁইয়া), a Bangla
dialect with tens of millions of speakers and almost no NLP resources.
One arm of a controlled ablation. Pipeline, benchmark and full results: https://github.com/A-K-M-Asifuzzaman/Chatgaiya-AI
chrF on the gold translation track of ChatgaiyyaBench, 120 fixed-seed items per direction.
| System | ctg → bn | bn → ctg |
|---|---|---|
| base (no adaptation) | 20.54 | 19.77 |
| A1_vocab_only | 56.69 | 49.26 |
| A2_subset_cpt | 59.62 | 51.04 |
| A3_full_cpt | 59.33 | 51.98 |
| B1_subset_ctg_syl | 62.89 | 50.00 |
| B2_subset_ctg_syl_bn | 65.64 | 48.64 |
| transducer (rules+lexicon, no GPU) | — | 57.94 |
| copy input unchanged | 29.51 | — |
Read the transducer row. A mined rule system costing no GPU time beats the neural arms at Bangla → Chittagonian. That is a real result about the difficulty of the task, reported rather than omitted.
An unadapted model is worse than doing nothing: base scores 20.54 on ctg → bn; copying the input scores 29.51.
| Adaptation | vocab+subset_cpt |
| Dialects in CPT | ctg, syl, bn |
| CPT tokens | 8,017,920 |
| SFT loss | 1.1264 |
| Wall time | 117 min |
| Added vocabulary | 3,914 Chittagonian subword tokens |
| Precision | fp16+GradScaler (Tesla T4, sm_75 has no hardware bf16) |
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
REPO = "AsifZaman1912/ChatgaiyyaLM-0.5B-B2_subset_ctg_syl_bn"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B", dtype=torch.float16)
model.resize_token_embeddings(len(tok)) # the adapter adds 3,914 tokens
model = PeftModel.from_pretrained(model, REPO).eval()
prompt = "চাটগাঁইয়া থেকে প্রমিত বাংলায় অনুবাদ করো:\nঅ্যাঁই ভাত ন হাইয়্যুম\nঅনুবাদ:"
ids = tok(prompt, return_tensors="pt")
out = model.generate(**ids, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
resize_token_embeddings before loading is required — the adapter carries a larger
embedding matrix than the base model. Always load the tokenizer from this repo:
token ids were assigned in insertion order, so a rebuilt tokenizer silently misaligns
every added token.
silver (auto-derived labels or transduced
text), awaiting native-speaker validation. Do not read absolute numbers off them.