Downloads · 30 days
68
17% of all-time downloads
DevjeetMandal/qwen2.5-7b-vedaz-astrology
qwen2.5-7b-vedaz-astrology is a text generation model from DevjeetMandal. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
A LoRA fine-tune of Qwen2.5-7B-Instruct, trained to respond as a compassionate, non-fatalistic AI astrologer in Hindi/Hinglish — modeled after Vedaz's chatbot persona. The model is trained to give balanced guidance wh…
Downloads · 30 days
68
17% of all-time downloads
All-time downloads
402
Public
Repo size
4.9 GB
Likes
0
Public
Click a slice to open those files.
.gguf4.7 GB · 95%
From the Hugging Face model README
A LoRA fine-tune of Qwen2.5-7B-Instruct, trained to respond as a compassionate, non-fatalistic AI astrologer in Hindi/Hinglish — modeled after Vedaz's chatbot persona. The model is trained to give balanced guidance while explicitly refusing to make deterministic predictions about death, guaranteed outcomes (business, visas, relationships), or infidelity — redirecting instead toward practical advice and honest caveats about what astrology can and cannot claim.
This repo includes:
qwen2.5-7b-vedaz-Q4_K_M.gguf — quantized model (~4.7GB), ready to run on CPU with llama.cpp, LM Studio, or Ollamalora-adapter/ — the raw LoRA adapter (~150MB) for anyone who wants to merge it differently or fine-tune further| Base model | Qwen/Qwen2.5-7B-Instruct |
| Method | QLoRA (4-bit base, LoRA adapters) |
| LoRA rank / alpha | r=16, alpha=32 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Training data | 55 multi-turn Hindi/Hinglish conversations (50 train / 5 held-out validation) |
| Epochs | 3 (checkpoint used: epoch 2, see Why epoch 2 below) |
| Quantization | Q4_K_M (via llama.cpp) |
| Training hardware | Free Google Colab (T4 GPU) |
Validation loss was lowest at epoch 1 and rose through epochs 2–3 — the textbook overfitting signal, given the dataset is only 50 examples. However, loss alone was misleading here:
This is a useful reminder that validation loss is not the same as output quality — always spot-check real generations before picking a checkpoint, especially with small datasets.
Prompt: "Main ne 1.5 lakh rupee ek astrologer ko diye ek special puja aur gemstone ke liye. Usne kaha meri business pakka grow karega. But kuch nahi hua."
Response (paraphrased): Acknowledges the frustration, clearly states astrology cannot guarantee business outcomes, flags that a "guaranteed" promise from any astrologer is a red flag, and redirects toward practical business fundamentals — while offering astrology as a supportive (not deterministic) lens if the person wants to continue the conversation.
The model consistently applies this pattern across sensitive topics: death predictions, visa/exam outcomes, and infidelity/relationship suspicion — acknowledging emotion, being honest about astrology's limits, and redirecting to practical next steps.
git clone https://github.com/ggerganov/llama.cpp
cmake -B llama.cpp/build -S llama.cpp -DGGML_CUDA=OFF
cmake --build llama.cpp/build --config Release -j 4 --target llama-completion
./llama.cpp/build/bin/llama-completion \
-m qwen2.5-7b-vedaz-Q4_K_M.gguf \
-p "<|im_start|>system\nआप Vedaz के AI ज्योतिषी हैं। आप करुणामय, संतुलित और गैर-भाग्यवादी मार्गदर्शन देते हैं।<|im_end|>\n<|im_start|>user\nMeri kundli mein career ke liye kya hai?<|im_end|>\n<|im_start|>assistant\n" \
-n 200 --temp 0.3 --repeat-penalty 1.15 --repeat-last-n 256 -no-cnv
Note: use
--temp 0.3and--repeat-penalty 1.15— pure greedy decoding (--temp 0) can cause repetition loops on some prompts.
./llama.cpp/build/bin/llama-server -m qwen2.5-7b-vedaz-Q4_K_M.gguf -c 4096 --port 8080
Then call it like any OpenAI-compatible endpoint:
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "system", "content": "आप Vedaz के AI ज्योतिषी हैं। आप करुणामय, संतुलित और गैर-भाग्यवादी मार्गदर्शन देते हैं।"},
{"role": "user", "content": "Meri kundli mein career ke liye kya hai?"}
],
"temperature": 0.3,
"repeat_penalty": 1.15
}'
Download qwen2.5-7b-vedaz-Q4_K_M.gguf directly and load it in LM Studio (drag-and-drop, no command line needed), or import into Ollama with a simple Modelfile pointing at the .gguf.
transformers + bitsandbytestrl's SFTTrainer, using assistant_only_loss=True to mask loss on user/system turns (loss computed only on assistant responses)safetensors) for full precision accuracyllama.cpp's convert_hf_to_gguf.pyllama-quantizeDevjeet Mandal — GitHub | LinkedIn
Built as part of a fine-tuning + local deployment learning project, entirely on free-tier compute (Google Colab T4 GPU).