YutaLM-M2-bnb-4bit

YutaLM-M2-bnb-4bit

YutaLM-M2-bnb-4bit is a lightweight, highly optimized 350M parameter hybrid architecture model fine-tuned specifically for Arabic Chat and Roleplay.

Developed under the LiteMind initiative, this model addresses the classic limitations of small-scale language models handling complex Arabic morphology and dialogue. By leveraging advanced target module tuning alongside direct training on Arabic linguistic embeddings (embed_tokens and lm_head), YutaLM-M2 delivers fluid, contextually expressive, and culturally nuanced Arabic interactions while maintaining an incredibly small hardware footprint.

---

🚀 Key Features

---

📝 Prompt Template & Format

The model utilizes the standard ChatML (LFM) sequence format. To achieve the best conversational or roleplaying stance, structure your inputs as follows:

<|im_start|>user
{Your Prompt Here}
<|im_end|>
<|im_start|>assistant

⚠️ Note: To prevent repetitive or broken generation artifacts common in smaller models, it is highly recommended to enforce a slight repetition penalty (1.05) and set a moderate temperature during inference.

---

💻 Quick Start & Inference

You can easily run this model using the Hugging Face transformers library. Ensure you have bitsandbytes and accelerate installed.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer

model_name = "LiteMind/YutaLM-M2-bnb-4bit"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Prepare the conversation
messages = [
    {"role": "user", "content": "مرحباً يا يوتا، كيف يمكنني برمجة نموذج ذكاء اصطناعي صغير؟"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True
).to("cuda")

# Initialize streamer for real-time output
streamer = TextStreamer(tokenizer, skip_prompt=True)

# Generate response
print("Assistant: ")
_ = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.73,
    top_k=50,
    top_p=0.9,
    repetition_penalty=1.05,
    streamer=streamer
)

---

⚙️ Development & Optimization

The model was meticulously fine-tuned using the Unsloth framework on a single-GPU instance.

To ensure the model didn't lose its linguistic capabilities during adaptation, a masked training approach was utilized to calculate loss exclusively on the assistant's rich Arabic responses. Additionally, vocabulary layers were dynamically updated to absorb Arabic stylistic semantics, preventing the typical degradation seen in smaller post-quantized models.

---

🛑 Limitations & Biases

• Due to its compact 350M parameter size, YutaLM-M2 should be treated as a specialized creative and conversational assistant rather than a factual encyclopedia.
• It may occasionally experience hallucinations if prompted with highly complex logical/mathematical problems.
• Performance is heavily optimized for Arabic; performance on multi-lingual switching or raw English coding might be limited compared to generalist base models.

---

📜 License

This model is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0). You are free to use, modify, and host this model, provided that any derivative works, modified versions, or web services leveraging this model are also open-sourced under the same AGPL-3.0 terms.

---

🤝 Acknowledgements

Special thanks to the Unsloth team for providing the memory-efficient frameworks that make fine-tuning hybrid, low-parameter models accessible and highly performant.