Downloads · 30 days
0
thegovind/azure-advisor-qwen25-0.5b-grpo
azure-advisor-qwen25-0.5b-grpo is a text generation model from thegovind. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Fine-tuned Qwen/Qwen2.5-0.5B-Instruct with SFT followed by Group Relative Policy Optimization (GRPO) to generate Azure Advisor-style recommendations.
Downloads · 30 days
0
Access
Public
Updated Mar 18, 2026
Repo size
46.7 MB
Likes
0
Public
Click a slice to open those files.
.safetensors35.2 MB · 69%
From the Hugging Face model README
Fine-tuned Qwen/Qwen2.5-0.5B-Instruct with SFT followed by Group Relative Policy Optimization (GRPO) to generate Azure Advisor-style recommendations.
This model builds on the SFT model with additional reward-based training using GRPO (rejection sampling + iterative SFT). It generates structured recommendations across 5 Azure Advisor categories:
| Iteration | Training Loss | Generation Avg Reward |
|---|---|---|
| 1 | 0.069 | 4.19 |
| 2 | 0.048 | 4.30 |
| 3 | 0.040 | 4.44 |
Pre-SFT baseline: 0.80/10
Post-SFT: 3.43/10 (+2.63)
GRPO iter losses: 0.069 -> 0.048 -> 0.040 (decreasing)
GRPO gen rewards: 4.19 -> 4.30 -> 4.44 (increasing)
HILL CLIMBING CONFIRMED
| Parameter | Value |
|---|---|
| Base Model | Qwen/Qwen2.5-0.5B-Instruct |
| Method | SFT + GRPO (rejection sampling) |
| LoRA Rank / Alpha | 16 / 32 |
| Quantization | 4-bit QLoRA (NF4) |
| GRPO Iterations | 3 |
| Samples per Prompt | 4 |
| Top-K Selection | 2 |
| Steps per Iteration | 30 |
| Learning Rate | 5e-5 |
| Hardware | NVIDIA RTX 3090 (24GB) |
| Function | Weight | Description |
|---|---|---|
| Format Compliance | 1.5 | Correct XML tags (<ANALYSIS>, <RECOMMENDATIONS>, <SUMMARY>) and valid JSON |
| Category Correctness | 2.0 | Valid Advisor categories (Cost, Security, Performance, etc.) |
| Grounding Quality | 2.0 | Claims supported by input evidence, no hallucinated resource IDs |
| Actionability | 2.0 | Concrete, feasible next steps with specific Azure actions |
| Completeness | 2.5 | Coverage of all issues with proper recommendation schema fields |
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen2.5-0.5B-Instruct",
torch_dtype=torch.float16,
device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "thegovind/azure-advisor-qwen25-0.5b-grpo")
tokenizer = AutoTokenizer.from_pretrained("thegovind/azure-advisor-qwen25-0.5b-grpo")
messages = [
{"role": "system", "content": "You are an Azure Advisor assistant. Analyze the workload and provide recommendations."},
{"role": "user", "content": "Analyze this Azure workload and provide recommendations..."}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
output = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))