Bharat-Tiny-LLM-v3.4

Bharat-Tiny-LLM v3.4

Status: experimental, superseded. Quality gates failed (GSM8K-Hindi ~13% vs v3's 56%, weak chat). Kept for reproducibility. For production use, see eulogik/Bharat-Tiny-LLM-v3 (+ v3.3 chat GGUFs in that repo). Details: v35_final_decision.json in the GitHub repo.

Mixed math+chat SFT (43,687 gold rows from eulogik/bharat-v3-mixed-sft-v34).

• Base: eulogik/Bharat-Tiny-LLM-v3 (1.7B)
• Method: LoRA r=16 (α=32) → merged fp16, 1 epoch, lr 2e-5, seq 1024
• Train loss: 1.82 → 1.23 (611 min, 2×T4 DataParallel on Kaggle)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "eulogik/Bharat-Tiny-LLM-v3.4",
    torch_dtype=torch.float16,
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained("eulogik/Bharat-Tiny-LLM-v3.4")

Training

See Kaggle kernel eulogikdevelopers/bharat-v34-mixed-sft.