qwen3-4b-bangla-agent

Qwen3-4B Bangla Agent

A Bangla-first assistant model for agentic use on CPU: it answers in Bangla, does grounded question answering over supplied documents (RAG) with citations, calls tools in Qwen3's native <tool_call> format from Bangla or English requests, and answers in Bangla after reading tool results.

It is a QLoRA fine-tune of Qwen/Qwen3-4B-Instruct-2507. The repository ships the LoRA adapter and merged GGUF files for llama.cpp, Ollama, LM Studio and Jan.

Repository layout

adapter/            LoRA adapter (safetensors) + tokenizer + chat template
gguf/               qwen3-4b-bangla-agent-Q4_K_M.gguf   2.4 GiB   default for CPU
                    qwen3-4b-bangla-agent-Q8_0.gguf     4.0 GiB   near-lossless
train_log.json      full Trainer log history (loss, LR, per-source eval loss)
last-checkpoint/    final trainer checkpoint (adapter + optimizer + scheduler), for resuming

Quick start

llama.cpp (CPU, recommended)

--jinja is required: it enables the Qwen3 chat template embedded in the GGUF, which is what turns the model's <tool_call> output into structured tool calls on the OpenAI-compatible API.

llama-server -m qwen3-4b-bangla-agent-Q4_K_M.gguf --jinja -c 8192 --port 8080

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="x")

tools = [{"type": "function", "function": {
    "name": "get_weather",
    "description": "Get the current weather for a city.",
    "parameters": {"type": "object",
                   "properties": {"city": {"type": "string", "description": "City name in English"}},
                   "required": ["city"]}}}]

r = client.chat.completions.create(
    model="local", tools=tools,
    messages=[{"role": "user", "content": "আজ সিলেট শহরের আবহাওয়া কেমন?"}])
print(r.choices[0].message.tool_calls)
# -> get_weather(arguments='{"city": "Sylhet"}')

Feed the tool result back as a tool message and the model answers in Bangla:

messages += [r.choices[0].message,
             {"role": "tool", "tool_call_id": r.choices[0].message.tool_calls[0].id,
              "content": '{"city": "Sylhet", "temperature_c": 29, "condition": "light rain", "humidity": 88}'}]
r2 = client.chat.completions.create(model="local", tools=tools, messages=messages)
print(r2.choices[0].message.content)
# -> সিলেট শহরে এখন তাপমাত্রা ২৯°C, আবহাওয়া হালকা বৃষ্টি এবং আর্দ্রতা ৮৮%।

Q4_K_M runs at roughly 10 to 20 tokens per second on a modern laptop CPU. Use Q8_0 when you want to rule out quantization as the cause of an odd answer.

Transformers + PEFT (GPU)

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_id = "Qwen/Qwen3-4B-Instruct-2507"
tok = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "zmsali/qwen3-4b-bangla-agent", subfolder="adapter").eval()

messages = [{"role": "user", "content": "বাংলাদেশের স্বাধীনতা যুদ্ধ কত সালে হয়েছিল? দুই বাক্যে বলো।"}]
text = tok.apply_chat_template(messages, tools=None, tokenize=False, add_generation_prompt=True)
ids = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=200, do_sample=False)
print(tok.decode(out[0, ids["input_ids"].shape[1]:], skip_special_tokens=True))

Pass tools=[...] to apply_chat_template for tool calling; the template renders the tool schemas into the system turn exactly as they were rendered during training.

Ollama

FROM ./qwen3-4b-bangla-agent-Q4_K_M.gguf
PARAMETER temperature 0.3

Add the TEMPLATE block from ollama show --modelfile qwen3:4b (same tool-call format), then ollama create bangla-agent -f Modelfile.

Prompt formats the model was trained on

System prompt. Half of the training conversations had a system prompt and half had none, so both work. Examples used in training:

• তুমি একজন সহায়ক বাংলা এআই সহকারী। ব্যবহারকারীর প্রশ্নের সঠিক ও সংক্ষিপ্ত উত্তর বাংলায় দাও।
• You are a helpful assistant. Answer in Bangla unless the user writes in another language.

Tool calling. Standard Qwen3 / Hermes format, produced by the chat template from OpenAI-style tools:

<tool_call>
{"name": "get_weather", "arguments": {"city": "Sylhet"}}
</tool_call>

Tool results go back as role: "tool" messages. The model was trained on single calls, parallel calls, multi-turn tool loops, and on conversations where tools were offered but not needed.

RAG. Put retrieved passages in the user message, numbered with Bangla digits, and end with the question:

নিচের নথিগুলো ব্যবহার করে প্রশ্নের উত্তর দাও। শুধুমাত্র নথিতে থাকা তথ্যের ভিত্তিতে উত্তর দেবে;
নথিতে উত্তর না থাকলে স্পষ্টভাবে বলবে যে তথ্যটি পাওয়া যায়নি।

[নথি ১]
...passage...

[নথি ২]
...passage...

প্রশ্ন: ...

The model answers briefly, often with a citation (নথি ১ অনুযায়ী, …), and declines when the passages do not contain the answer (প্রদত্ত নথিতে এই প্রশ্নের উত্তর পাওয়া যায়নি।). For agentic RAG, expose retrieval as a tool named search_documents(query, top_k); the model was trained to call it.

Training data

All examples were rendered through the base model's own chat template with tools= where applicable, so the training text is byte-identical to inference text. Loss was computed only on assistant turns, including <tool_call> blocks and the closing <|im_end|>. Examples longer than 2,048 tokens were dropped. Approximately 16.1k conversations were used for training; 100 per source (40 for the smallest) were held out for evaluation.

Bangla is token-expensive in Qwen's vocabulary (about 1.5 characters per token); a Bangla-Instruct row averages ~1,000 tokens and a RAG row ~1,300.

Training procedure

The merged GGUF files were produced by merging the adapter into fp16 base weights, converting with llama.cpp convert_hf_to_gguf.py --outtype f16, and quantizing with llama-quantize to Q4_K_M and Q8_0. Because the adapter was trained against 4-bit weights and merged into fp16 ones, there is a small, expected mismatch; Q8_0 is the closer of the two to the adapter as trained.

Evaluation

Held-out cross-entropy loss per data source, before training and at three points during it. Lower is better. The tools row is the one to watch for forgetting: it fell rather than rose, so Bangla training did not erode tool calling.

Qualitative checks, greedy decoding, run on the 4-bit base plus adapter after training. Verbatim outputs:

No standardized Bangla benchmark scores are reported. The eval losses above are on held-out slices of the training distribution and are not comparable across models.

Limitations

• Factual recall in Bangla is weak, as expected for a 4B model, and the last example above shows a confident hallucination. Use the model with retrieval or a search tool for facts; do not rely on its memory.
• Bangla-Instruct is machine-generated and noisy. Some training responses are glossaries or truncated answers, and this shows in occasional awkward phrasing (the second sentence of the first example).
• Synthetic tool-calling data is narrow. Seven tools, templated phrasing, twenty cities. The model generalizes to unseen schemas because the Hermes data is diverse, but Bangla answers after tool results tend to echo the training templates.
• Standard Bangla only. Regional dialects (Chittagonian, Sylheti, Noakhali and others) were not in the training data.
• Non-thinking model. The base is the 2507 Instruct checkpoint and emits no reasoning traces. Do not set enable_thinking.
• Trained at 2,048 tokens. Longer contexts inherit the base model's behavior and were not tested.
• No safety tuning beyond the base model's. Qwen3's refusals carry over; nothing Bangla-specific was added.

Intended use

Local Bangla assistants and agents on CPU: document question answering over a private corpus, tool-using assistants (weather, calculators, reminders, search), and as a starting adapter for further Bangla fine-tuning. Not intended for medical, legal or financial advice, or for any use where a wrong Bangla fact stated confidently is harmful.

License and attribution

The base model and this adapter are released under Apache-2.0. Note that one training source, squad_bn, is licensed CC-BY-NC-SA-4.0; if you need a strictly commercial-clean model, retrain with the RAG source removed or replaced (the training scripts make this a one-line change). Bangla-Instruct is MIT and hermes-function-calling-v1 is Apache-2.0.

If you use this model, please also credit the data sources:

• Raihan and Zampieri, TigerLLM: A Family of Bangla Large Language Models, ACL 2025 (Bangla-Instruct)
• Bhattacharjee et al., BanglaBERT, Findings of NAACL 2022 (squad_bn)
• NousResearch, Hermes Function Calling v1

Reproduction

Training was done with three scripts run from a single Kaggle notebook: a data builder that renders and masks every source through the chat template, a torchrun DDP trainer that pushes full checkpoints to this repository every 100 steps for resumable training, and an export cell that merges and converts to GGUF. train_log.json and last-checkpoint/trainer_state.json contain the complete run history. Random seed 3407 throughout.

Citation

@misc{qwen3-4b-bangla-agent,
  author = {Shahjahan Ali},
  title  = {Qwen3-4B Bangla Agent: a Bangla tool-calling and RAG assistant for CPU},
  year   = {2026},
  url    = {https://huggingface.co/zmsali/qwen3-4b-bangla-agent}
}