Vedaz-Qwen-Finetuned

Vedaz Qwen2.5 Finetuned (LoRA)

A LoRA fine-tuned version of Qwen2.5-3B-Instruct for the Vedaz AI internship assessment.

The model is trained to simulate a compassionate Vedic astrology assistant that:

• Provides balanced and empathetic guidance.
• Requests birth details before horoscope analysis.
• Avoids deterministic or fear-based predictions.
• Never predicts death or guaranteed future events.
• Encourages practical decision making alongside astrological guidance.

---

Model Details

---

Dataset

The model was fine-tuned on the conversation dataset provided as part of the Vedaz AI internship assessment.

Dataset characteristics:

• 55 multi-turn conversations
• System/User/Assistant format
• Hindi + Hinglish
• Safety-focused responses
• Kundli analysis workflow
• Empathetic conversational style

The dataset includes conversations related to:

• Career
• Marriage
• Relationships
• Finance
• Mental health safety
• General astrology guidance

---

Training

Fine-tuning

• LoRA Fine-tuning
• Unsloth
• PEFT
• TRL SFTTrainer

Hyperparameters

Approximately 1.9% of the model parameters were trained using LoRA adapters.

---

Intended Use

Suitable for:

• Astrology chatbot demos
• Conversational AI experiments
• Fine-tuning demonstrations
• Internship assignments
• Educational purposes

---

Limitations

Because the training dataset contains only 55 conversations, the model has several limitations:

• May repeat phrases.
• May overfit training patterns.
• Does not possess real astrological reasoning.
• Responses should not be considered professional astrological advice.
• Should not be used for medical, legal, or financial decision making.

---

Safety

The model is designed to:

• Avoid fatalistic predictions.
• Avoid guarantees about the future.
• Respond empathetically.
• Request birth details before horoscope analysis.
• Encourage practical action alongside astrology.

---

Example

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "moksh0987654/Vedaz-Qwen-Finetuned"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

messages = [
    {"role":"system","content":"You are Vedaz AI astrologer."},
    {"role":"user","content":"Meri job kab lagegi?"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    temperature=0.7
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Repository

GitHub:

https://github.com/0001Moksh/Vedaz-Qwen-Finetuning

---

Author

Moksh Bhardwaj

AI & Machine Learning Engineer

GitHub: https://github.com/0001Moksh

LinkedIn: https://linkedin.com/in/moksh-bhardwaj-0001moksh

Portfolio: https://mokshbhardwaj.netlify.app

---

Framework Versions

• Transformers
• PEFT 0.19.1
• TRL
• Unsloth
• Accelerate
• BitsAndBytes