krishna-gita-gguf

krishna-gita-gguf

A LoRA fine-tune of Qwen2.5-3B-Instruct trained on 180 hand-authored Krishna/Arjuna dialogue pairs, grounded in Edwin Arnold's public-domain 1885 English translation of the Bhagavad Gita — fused into the base model and exported to GGUF for portable (non-MLX) inference.

Built with Qwen. Base model licensed under the Qwen RESEARCH LICENSE AGREEMENT (non-commercial / research-and-evaluation use only) — see LICENSE.txt and NOTICE.txt in this repo, and https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE.

What this is

• Teaches voice, not facts. This adapter was trained to reproduce Krishna's cadence and rhetorical style — direct address, calm authority, metaphor — not to memorize Gita content precisely. Used alone (without retrieval grounding), it will confidently answer questions the Gita never addresses, in-voice but doctrinally invented. See the source repo's RAG pipeline (src/rag_persona.py) for the intended grounded-generation setup.
• Small dataset. 180 examples (162 train / 18 valid) is on the low end of what's needed for robust generalization. Expect some inconsistency on questions far from the training distribution.
• Checkpoint selected by validation loss, not the final training iteration — see the source repo's notebook for the full loss curve and why that mattered here.

Files

• krishna-3b-f16.gguf — full-precision GGUF (~6.2GB)
• krishna-3b-q4_k_m.gguf — Q4_K_M quantized GGUF (~1.9GB), recommended for CPU inference

Usage (llama-cpp-python)

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="<this-repo>",
    filename="krishna-3b-q4_k_m.gguf",
)
out = llm.create_chat_completion(messages=[
    {"role": "system", "content": "You are Krishna, speaking to Arjuna on the battlefield of Kurukshetra. Speak with calm authority, address him directly, and teach through his doubt rather than dismissing it."},
    {"role": "user", "content": "Why shouldn't Arjuna grieve for his kinsmen?"},
])
print(out["choices"][0]["message"]["content"])

Build pipeline

1. LoRA fine-tuned with MLX-LM (Apple Silicon) — mlx_lm.lora
2. Best checkpoint selected by validation loss (not final iteration)
3. Fused into base weights + dequantized — mlx_lm.fuse --dequantize
4. Converted to GGUF F16 — llama.cpp's convert_hf_to_gguf.py
5. Quantized to Q4_K_M — llama_cpp.llama_model_quantize

Full source, training data, and RAG pipeline: https://github.com/aditibhoge1-ops/krishna-rag-companion

Non-commercial use only

Per the Qwen RESEARCH LICENSE AGREEMENT, this model (including this fine-tune) may only be used for research or evaluation purposes. Commercial use requires a separate license from Alibaba Cloud.