Bharat-Tiny-LLM-GGUF

Bharat-Tiny-LLM (GGUF)

llama.cpp builds of Bharat-Tiny-LLM — India's first native edge AI for Hinglish & Hindi. These run cross-platform: Android, Raspberry Pi, CPU, and GPU via llama.cpp / llama-cpp-python.

Built by eulogik

Files

Quick start

pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="bharat-tiny-llm-q4_k_m.gguf", n_ctx=1024)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "Chai peete hain?"}],
    temperature=0.3, top_p=0.85, max_tokens=256, repeat_penalty=1.25,
)["choices"][0]["message"]["content"])

⚠️ Generation config matters. The base Qwen2.5-1.5B emits garbled out-of-script tokens at high temperature. Always use temperature ≈ 0.3 + repeat_penalty ≥ 1.25.

Other builds

Links

• 🤗 Edge model (MLX): https://huggingface.co/eulogik/Bharat-Tiny-LLM
• 🤗 fp16 fused: https://huggingface.co/eulogik/Bharat-Tiny-LLM-fused
• 🚀 Demo: https://huggingface.co/spaces/eulogik/bharat-tiny-llm-v3-demo
• 💻 Source: https://github.com/eulogik/Bharat-Tiny-LLM
• 📦 PyPI: https://pypi.org/project/bharat-tiny-llm/
• 🏢 Built by eulogik

License

Apache-2.0 (base Qwen2.5-1.5B weights Apache-2.0; LoRA adapter Apache-2.0).