Downloads · 30 days
0
Srstilm-Indic-4B-CPT
sunilprakash/Srstilm-Indic-4B-CPT
Srstilm-Indic-4B-CPT is a text generation model from sunilprakash. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Indic QLoRA continued-pretrain of Qwen3.5-4B-Base with a +29K-token Indic tokenizer extension (277K vocab). Trained ~13M tokens on a 400K-sequence Indic+code+English corpus (50/30/20); embeddings frozen, LoRA on attn/…
Access
Public
Updated Jun 29, 2026
Repo size
6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors6 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Jun 28, 2026
- Updated
- Jun 29, 2026
- SHA
- 461bbc99
Base models
Model card
From the Hugging Face model README
Srstilm-Indic-4B-CPT
Indic QLoRA continued-pretrain of Qwen3.5-4B-Base with a +29K-token Indic tokenizer extension (277K vocab). Trained ~13M tokens on a 400K-sequence Indic+code+English corpus (50/30/20); embeddings frozen, LoRA on attn/MLP. On held-out Indic text it lowers loss 9.16 to 6.96 (24%) vs the extended-vocab base. Cost-effective experimental build. Use with the extended tokenizer in this repo.
Model
- License: apache-2.0
- Method: continued pretraining with an extended tokenizer (+Indic merges), Sailor new-row embedding init, clean base-row freeze.
- Languages: hi, mr, ta, te, kn, sa, en
Evaluation
| Benchmark | Score |
|---|---|
| Held-out Indic loss (base + extended tokenizer) | 9.16 |
| Held-out Indic loss (this model, + LoRA) | 6.96 |
| Held-out Indic loss reduction (%) | 24 |
Intended use & limitations
Research / Indic + code generation. As a continued-pretrained base it inherits Qwen/Qwen3.5-4B-Base's strengths and biases; evaluate before production use.
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Languages
- hi, mr, ta, te, kn, sa
- Created
- Jun 28, 2026
- Updated
- Jun 29, 2026