Downloads · 30 days
27
5% of all-time downloads
ihumaunkabir/mistral_bangla_q4_k_m_gguf
mistral_bangla_q4_k_m_gguf is a text generation model from ihumaunkabir. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
A standalone, ready-to-run quantized Mistral-7B model adapted for Bangla (Bengali). The LoRA adapters were merged into the base weights and quantized to Q4KM GGUF format -- no base model loading required, just point O…
Downloads · 30 days
27
5% of all-time downloads
All-time downloads
499
Public
Repo size
8.7 GB
Likes
0
Public
Click a slice to open those files.
.gguf4.4 GB · 100%
From the Hugging Face model README
A standalone, ready-to-run quantized Mistral-7B model adapted for Bangla (Bengali). The LoRA adapters were merged into the base weights and quantized to Q4_K_M GGUF format -- no base model loading required, just point Ollama / llama.cpp at the file and run.
Trained via a two-stage recipe: continual pre-training (CPT) on Bangla Wikipedia, then
supervised fine-tuning (SFT) on the
ihumaunkabir/alpaca-gpt4-bangla
instruction dataset, using Unsloth.
Looking for the trainable adapters? The PEFT LoRA adapters (to continue fine-tuning or re-quantize) live at
ihumaunkabir/mistral_bangla_lora.
| detail | value |
|---|---|
| File size | ~4.4 GB |
| Quantization | Q4_K_M (recommended balance of quality + size) |
| Base model needed | No -- weights are merged in |
| Inference tools | Ollama, llama.cpp, LM Studio |
# Pull directly from Hugging Face
ollama pull hf.co/ihumaunkabir/mistral_bangla_q4_k_m_gguf:Q4_K_M
# Run interactively
ollama run hf.co/ihumaunkabir/mistral_bangla_q4_k_m_gguf:Q4_K_M
Or build a local Modelfile with the Alpaca template (see "Prompt format" below) and the system prompt of your choice, then:
ollama create mistral-bangla -f Modelfile
ollama run mistral-bangla
# Clone + build llama.cpp, then:
./llama-cli \
-m mistral-7b-v0.3.Q4_K_M.gguf \
-p "নিচে একটি নির্দেশনা দেওয়া আছে..." \
-n 128 --color
Download the .gguf file and load it through the GUI. No conversion needed.
This model was trained on the Alpaca template, translated to Bangla. At inference time, leave the response blank and let the model generate:
নিচে একটি নির্দেশনা দেওয়া আছে, যা একটি কাজের বর্ণনা দেয়। অনুরোধটি যথাযথভাবে সম্পূর্ণ করে একটি উত্তর লিখুন।
### নির্দেশনা:
<instruction here>
### উত্তর:
<response here>
For an Ollama Modelfile, wrap it as:
TEMPLATE """{{ if .Prompt }}নিচে একটি নির্দেশনা দেওয়া আছে, যা একটি কাজের বর্ণনা দেয়। অনুরোধটি যথাযথভাবে সম্পূর্ণ করে একটি উত্তর লিখুন।
### নির্দেশনা:
{{ .Prompt }}
### উত্তর:
{{ end }}{{ .Response }}"""
PARAMETER stop "### নির্দেশনা:"
PARAMETER stop "### উত্তর:"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
This GGUF was produced by:
unsloth/mistral-7b-v0.3 in 4-bit (QLoRA).wikimedia/wikipedia, config 20231101.bn).ihumaunkabir/alpaca-gpt4-bangla (49,969 Bangla instruction pairs).LoRA config: rank 128, alpha 32, rsLoRA on, targets include embed_tokens + lm_head.
max_steps=120) -- quality will be rough until full epochs are run.If you use this model, please cite both this GGUF repo and the alpaca-gpt4-bangla dataset,
plus the base Mistral model.
@misc{mistral-bangla-gguf,
author = {ihumaunkabir},
title = {mistral-bangla: Bangla Mistral-7B in Q4_K_M GGUF format},
year = {2026},
url = {https://huggingface.co/ihumaunkabir/mistral_bangla_q4_k_m_gguf},
note = {Merged LoRA + Q4_K_M quantization of CPT + SFT Mistral-7B}
}
@misc{mistral-bangla-lora,
author = {ihumaunkabir},
title = {mistral-bangla: Bangla CPT + SFT LoRA adapters for Mistral-7B-v0.3},
year = {2026},
url = {https://huggingface.co/ihumaunkabir/mistral_bangla_lora}
}
@misc{alpaca-gpt4-bangla,
author = {ihumaunkabir},
title = {alpaca-gpt4-bangla: A Bangla instruction-following dataset},
year = {2026},
url = {https://huggingface.co/datasets/ihumaunkabir/alpaca-gpt4-bangla},
note = {Machine translation (Korean -> Bangla) of FreedomIntelligence/alpaca-gpt4-korean}
}
@misc{mistral7b,
author = {Mistral AI},
title = {Mistral-7B-v0.3},
year = {2024},
url = {https://huggingface.co/mistralai/Mistral-7B-v0.3}
}