lumichats_4Bz_v1.4

LumiChats 4B v1.4

<div align="center">

Premium AI-powered conversational model optimized for dialogue generation

🚀 Try LumiChats Cloud | 📖 Documentation | 💬 Community

</div>

---

Table of Contents

• Overview
• Model Details
• Key Features
• Quick Start
• Usage Examples
• Training Details
• Benchmark Performance
• Limitations
• About LumiChats
• License
• Citation
• Acknowledgements

---

Overview

LumiChats 4B v1.4 is a fine-tuned conversational AI model based on Google's Gemma-3-4B-IT, specifically optimized for natural dialogue generation and multi-turn conversations. This model has been adapted using advanced Parameter-Efficient Fine-Tuning (PEFT) techniques with LoRA and 4-bit quantization to deliver premium conversational capabilities while maintaining exceptional resource efficiency.

Built for students, developers, and creators who need powerful AI assistance without the overhead of expensive infrastructure, LumiChats 4B v1.4 combines the robust foundation of Gemma-3 with specialized training on conversational data to deliver responses that are contextually aware, coherent, and engaging.

What Makes This Model Special?

• ✨ Specialized Conversational Training: Fine-tuned on 100k high-quality dialogue examples from the FineTome dataset
• 🚀 Ultra-Efficient: 4-bit quantization reduces memory footprint by 70-80% with minimal performance impact
• 🎯 Response-Focused Learning: Trained using train_on_responses_only technique for superior output quality
• 💪 Production-Ready: Only 0.35% of parameters trained via LoRA, resulting in fast inference and small model size
• 🔧 Optimized with Unsloth: 2x faster training and inference with smart gradient offloading

---

Model Details

Core Specifications

Architecture

LumiChats 4B v1.4 inherits the robust Gemma-3 transformer architecture with the following enhancements:

• LoRA Rank (r): 8
• LoRA Alpha: 8
• LoRA Dropout: 0 (no dropout applied)
• Bias Adaptation: None (only weight matrices adapted)
• Tuned Components:
  • Language processing layers
  • Multi-head attention modules
  • Multi-layer perceptron (MLP) modules

Technical Advantages

1. Memory Efficiency: 4-bit quantization reduces VRAM requirements from ~16GB to ~4GB
2. Fast Fine-Tuning: LoRA enables training in minutes rather than hours
3. Small Adapter Size: Only LoRA adapters need to be saved (~60MB vs 8GB full model)
4. Quality-Focused Training: Response-only loss calculation ensures high-quality outputs
5. Production Performance: 2x speedup with Unsloth optimizations

---

Key Features

🎯 Conversational Excellence

• Natural, context-aware dialogue generation
• Multi-turn conversation support
• Instruction-following capabilities
• Consistent tone and personality

🧠 Advanced Capabilities

• Question answering with detailed explanations
• Code generation and debugging assistance
• Content creation (essays, articles, creative writing)
• Summarization of complex topics
• Multilingual support (140+ languages)

⚡ Performance Optimized

• 4-bit quantization for reduced memory usage
• Runs on consumer GPUs (RTX 3060, T4, etc.)
• Fast inference with Unsloth optimizations
• Compatible with GGUF format for llama.cpp deployment

---

Quick Start

Installation

First, install the required dependencies:

pip install -U unsloth transformers trl bitsandbytes accelerate peft

Basic Usage

Loading the Model

from unsloth import FastLanguageModel
from transformers import TextStreamer

# Load model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="adityakum667388/lumichats_4Bz_v1.4",
    max_seq_length=2048,
    load_in_4bit=True,
    dtype=None,  # Auto-detection
)

# Enable fast inference mode
FastLanguageModel.for_inference(model)

Simple Text Generation

# Prepare your prompt
messages = [
    {"role": "user", "content": "Explain quantum computing in simple terms."}
]

# Apply chat template
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_tensors="pt",
    return_dict=True,
).to("cuda")

# Generate response
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    top_k=50,
    use_cache=True,
)

# Decode and print
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Streaming Generation

# For real-time streaming output
streamer = TextStreamer(tokenizer, skip_prompt=True)

messages = [
    {"role": "user", "content": "Write a short poem about artificial intelligence."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_tensors="pt",
    return_dict=True,
).to("cuda")

_ = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.8,
    top_p=0.95,
    streamer=streamer,
)

---

Usage Examples

Multi-Turn Conversation

# Build conversation history
conversation = [
    {"role": "user", "content": "What is machine learning?"},
    {"role": "assistant", "content": "Machine learning is a subset of artificial intelligence that enables computers to learn and improve from experience without being explicitly programmed. It uses algorithms to analyze data, identify patterns, and make decisions with minimal human intervention."},
    {"role": "user", "content": "Can you give me a practical example?"}
]

inputs = tokenizer.apply_chat_template(
    conversation,
    add_generation_prompt=True,
    tokenize=True,
    return_tensors="pt",
    return_dict=True,
).to("cuda")

outputs = model.generate(**inputs, max_new_tokens=300)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Code Generation

messages = [
    {"role": "user", "content": "Write a Python function to calculate the Fibonacci sequence up to n terms."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_tensors="pt",
    return_dict=True,
).to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.3,  # Lower temperature for more deterministic code
    top_p=0.9,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Creative Writing

messages = [
    {"role": "user", "content": "Write a creative short story opening about a time traveler discovering an ancient civilization."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_tensors="pt",
    return_dict=True,
).to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=400,
    temperature=0.9,  # Higher temperature for creativity
    top_p=0.95,
    top_k=64,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Using GGUF Format (llama.cpp)

For deployment with llama.cpp or other GGUF-compatible tools:

# Download the GGUF version
# Available at: adityakum667388/lumichats_4B_v1.4-gguf

# Run with llama.cpp
./llama-cli -m lumichats-4b-v1.4-Q4_K_M.gguf -p "Your prompt here" -n 512

---

Training Details

Dataset

Training Data: mlabonne/FineTome-100k

• 100,000 high-quality conversational examples
• ShareGPT-style multi-turn dialogues
• Diverse topics covering general knowledge, reasoning, and creative tasks
• Formatted using Gemma-3 chat template

Data Preprocessing

1. Format Standardization: Converted all examples to consistent {role, content} format
2. Chat Template Application: Applied Gemma-3-specific conversation markers
3. BOS Token Handling: Removed redundant <bos> tokens to prevent duplication
4. Response-Only Training: Masked user input tokens (set labels to -100) to focus learning on assistant responses

Training Configuration

Training Environment

• Hardware: NVIDIA Tesla T4 GPU (Google Colab)
• Framework: Unsloth (optimized Transformers + TRL)
• Peak Memory Usage: ~9.2 GB VRAM
• Training Time: ~7 minutes (30 steps demonstration)
• CUDA Version: 12.1
• Precision: 4-bit (NF4) for weights, BF16 for computation

LoRA Configuration

{
    "r": 8,                          # LoRA rank
    "lora_alpha": 8,                 # LoRA scaling factor
    "lora_dropout": 0,               # No dropout
    "bias": "none",                  # Only adapt weights
    "task_type": "CAUSAL_LM",
    "target_modules": [
        "q_proj", "k_proj", "v_proj",  # Attention
        "o_proj", "gate_proj",          # MLP
        "up_proj", "down_proj"
    ]
}

Key Training Innovations

1. train_on_responses_only: Calculates loss only on model outputs, preventing memorization of user inputs
2. Smart Gradient Offloading: Unsloth automatically manages gradient memory
3. 8-bit AdamW: Reduces optimizer state memory by 75%
4. Gradient Accumulation: Simulates larger batch sizes without OOM errors

---

Benchmark Performance

Conversational Quality

LumiChats 4B v1.4 inherits the strong performance of Gemma-3-4B-IT and improves specifically on conversational metrics:

Base Model Capabilities (Inherited)

From Gemma-3-4B-IT foundation:

Reasoning & Factuality

• HellaSwag: 77.2% (10-shot)
• PIQA: 79.6% (0-shot)
• TriviaQA: 65.8% (5-shot)
• ARC-Challenge: 56.2% (25-shot)

STEM & Code

• MMLU: 59.6% (5-shot)
• GSM8K: 38.4% (8-shot)
• HumanEval: 36.0% (0-shot)
• MBPP: 46.0% (3-shot)

Multilingual

• MGSM: 34.7%
• Global-MMLU-Lite: 57.0%

Note: Fine-tuning focused on conversational ability; general benchmark scores remain comparable to base model.

---

Limitations

Known Limitations

1. Factual Accuracy: Like all language models, may generate plausible-sounding but incorrect information
2. Temporal Knowledge: Training data cutoff at January 2025; no awareness of events after this date
3. Mathematical Reasoning: While capable, complex mathematical proofs may be challenging
4. Quantization Trade-offs: 4-bit quantization may slightly reduce precision in some edge cases
5. Context Length: While supporting 128K tokens, performance may degrade with very long contexts
6. Multilingual Performance: Optimized for English; other languages supported but with varying quality

Responsible Use

• Bias Awareness: Model may reflect biases present in training data
• Verification: Always verify critical information from authoritative sources
• Privacy: Do not share sensitive personal information with the model
• Content Safety: Review outputs before sharing publicly or in production systems
• Use Cases: Best suited for general assistance, learning, and creative tasks

Not Recommended For

• ❌ Medical diagnosis or treatment decisions
• ❌ Legal advice or formal legal interpretations
• ❌ Financial investment decisions
• ❌ Safety-critical systems or decisions
• ❌ Generating content that violates laws or ethical guidelines

---

About LumiChats

Our Mission

LumiChats is dedicated to democratizing access to premium AI technology at affordable prices. We believe powerful AI should be accessible to students, developers, and creators worldwide—without expensive subscriptions or complex infrastructure.

LumiChats Platform Features

🌟 Pay-Per-Day Pricing: Only ₹69/day on days you use it—perfect for students and occasional users

🤖 39+ AI Models: Access Claude, GPT-4, Gemini, DeepSeek, Qwen, Mistral, and more from one platform

📚 Study Mode: Page-by-page PDF learning with custom quizzes and instant note generation

🧠 Memory Control: Selective context management to prevent topic confusion

💬 5M Tokens Daily: Generous daily limits across all premium and open-source models

🔧 Zero Setup: No infrastructure, no GPU costs, no technical complexity

Why We Built This Model

We created LumiChats 4B v1.4 as part of our commitment to open-source AI. While our cloud platform offers 39+ models with advanced features, we believe in giving developers and researchers:

• Free Alternative: Download and run locally at zero cost
• Full Control: Self-host on your own hardware
• Learning Resource: Study our fine-tuning approach
• Research Foundation: Build on our work for your own projects

Commercial Use

LumiChats 4B v1.4 is released under the Gemma License, which permits commercial use. If you use this model in production:

• ✅ Commercial applications allowed
• ✅ Modification and redistribution permitted
• ✅ Attribution appreciated but not required
• ⚠️ Review the full Gemma License for details

Connect With Us

• 🌐 Website: lumichats.com
• 💬 Discord: Join our community (coming soon)
• 📧 Email: [email protected]
• 🐦 Twitter: @lumichats (coming soon)

Try LumiChats Cloud

Ready for the full experience? Our cloud platform offers:

• Instant access to 39+ AI models
• Study Mode with page-by-page PDF learning
• Memory control across subjects
• Image analysis and quiz generation
• No setup, no infrastructure management

Start Free: lumichats.com • No credit card required • ₹69/day only when active

---

License

This model is licensed under the Gemma License by Google DeepMind.

• Base Model License: Gemma Terms of Use
• Fine-tuning & LoRA Adapters: Same Gemma License applies
• Commercial Use: ✅ Permitted
• Redistribution: ✅ Allowed with attribution

By using this model, you agree to comply with the Gemma License terms.

---

Citation

If you use LumiChats 4B v1.4 in your research or applications, please cite:

This Model

@misc{lumichats4b2025,
  title={LumiChats 4B v1.4: Fine-tuned Gemma-3 for Conversational AI},
  author={LumiChats Team},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/adityakum667388/lumichats_4Bz_v1.4}}
}

Base Model (Gemma-3)

@article{gemma_2025,
  title={Gemma 3},
  url={https://goo.gle/Gemma3Report},
  publisher={Kaggle},
  author={Gemma Team},
  year={2025}
}

---

Acknowledgements

This project builds on the exceptional work of multiple teams and open-source communities:

Core Technologies

• Google DeepMind: For the incredible Gemma-3 model family
• Unsloth: For their fast, memory-efficient fine-tuning framework
• Hugging Face: For Transformers, TRL, and Datasets libraries
• mlabonne: For the high-quality FineTome-100k dataset

Special Thanks

• Tim Dettmers: For bitsandbytes (4-bit quantization)
• Edward Hu et al.: For the LoRA paper and implementation
• The Open-Source Community: For continuous improvements and support

Infrastructure

Training was conducted on Google Colab's free tier with NVIDIA Tesla T4 GPUs, demonstrating that high-quality fine-tuning is accessible to everyone.

---

Future Roadmap

• Quantized GGUF versions (Q4_K_M, Q8_0)
• Extended context fine-tuning (up to 128K)
• Multi-modal capabilities exploration
• Domain-specific fine-tunes (code, creative writing)

---

Contributing

We welcome contributions! If you've:

• Improved the model with additional fine-tuning
• Created useful tools or integrations
• Found bugs or limitations
• Developed creative use cases

Please share your work with the community through discussions or pull requests.

---

<div align="center">

Made with ❤️ by the LumiChats Team

Democratizing AI, one model at a time

🚀 Try LumiChats Cloud • 📖 Docs • 💬 Community

</div>