Downloads · 30 days
12
11% of all-time downloads
thegovind/azure-architect-qwen35-0.8b-grpo-merged
azure-architect-qwen35-0.8b-grpo-merged is a text generation model from thegovind. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
A fully merged Qwen 3.5 0.8B model trained with GRPO (Group Relative Policy Optimization) to be an Azure Cloud Solution Architect with structured reasoning capabilities. This is the LoRA adapters merged into the base…
Downloads · 30 days
12
11% of all-time downloads
All-time downloads
108
Public
Parameters
752M
3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3 GB · 99%
From the Hugging Face model README
A fully merged Qwen 3.5 0.8B model trained with GRPO (Group Relative Policy Optimization) to be an Azure Cloud Solution Architect with structured reasoning capabilities. This is the LoRA adapters merged into the base weights — ready for deployment with no adapter loading needed.
<REASONING>...</REASONING> and <SOLUTION>...</SOLUTION> formatQuestion: Which Azure service handles global load balancing?
A. Azure Load Balancer B. Azure Front Door C. Traffic Manager D. Application Gateway
<REASONING>
Azure Front Door provides global HTTP/HTTPS load balancing with built-in CDN,
WAF, and SSL offloading. It operates at Layer 7 and routes traffic to the
closest healthy backend across regions. Azure Load Balancer is regional (Layer 4),
Traffic Manager is DNS-based (slower failover), and Application Gateway is
regional Layer 7. For global load balancing with low latency, Front Door is ideal.
</REASONING>
<SOLUTION>B</SOLUTION>
| Parameter | Value |
|---|---|
| Base Model | unsloth/Qwen3.5-0.8B |
| Method | SFT → GRPO with GSPO variant (loss_type=dr_grpo), then merged |
| LoRA Rank | 16 |
| SFT Dataset | thegovind/azure-architecture-vqa (1,678 train examples) |
| GRPO Dataset | thegovind/azure-architecture-grpo-benchmark (200 train / 51 eval) |
| SFT Training | 42.6 min, 210 steps, loss 0.6517 |
| GRPO Training | ~4 hr 40 min, 200 steps, peak reward 5.5/7.0 |
| Hardware | 1x NVIDIA RTX 4090 (24GB) |
| Total Cost | ~$1.88 on vast.ai |
| Signal | Max Score | What It Measures |
|---|---|---|
| R1 — Format Compliance | +2.0 | Proper <REASONING> and <SOLUTION> XML tags |
| R2 — Answer Correctness | +3.0 | Exact match on A/B/C/D answer letter |
| R3 — Azure Relevance | +1.0 | Mentions relevant Azure services in reasoning |
| R4 — Reasoning Quality | +1.0 | Substantive reasoning (50–500 words) |
| Total | 7.0 |
!pip install -q --upgrade transformers accelerate torch
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"thegovind/azure-architect-qwen35-0.8b-grpo-merged",
torch_dtype=torch.float16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("thegovind/azure-architect-qwen35-0.8b-grpo-merged")
SYSTEM_PROMPT = (
"You are an expert Azure Cloud Solution Architect. "
"Provide reasoning in <REASONING></REASONING> tags, "
"then your answer in <SOLUTION></SOLUTION> tags."
)
question = """Which Azure service is best for real-time fraud detection at scale?
A. Azure Batch
B. Azure Stream Analytics with Event Hubs
C. Azure SQL Database
D. Azure Blob Storage"""
prompt = f"<|im_start|>system\n{SYSTEM_PROMPT}<|im_end|>\n<|im_start|>user\n{question}\n\nProvide reasoning in <REASONING></REASONING> tags and answer in <SOLUTION></SOLUTION> tags.<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
output = model.generate(**inputs, max_new_tokens=1024, temperature=0.7, do_sample=True)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
| Version | Use When |
|---|---|
| This (merged) | Deployment, inference servers, GGUF conversion, Foundry Local, vLLM |
| LoRA | Further fine-tuning, experimentation, saving storage (43 MB vs 1.6 GB) |
Stage 1: SFT — "Learn Azure knowledge from 1,678 Q&A pairs"
→ Supervised Fine-Tuning on Azure Architecture Center content
Stage 2: GRPO — "Learn to reason through problems via RL with 4 reward signals"
→ Reinforcement Learning with structured output format
Merge — LoRA adapters merged into base weights for easy deployment
→ This model
| Resource | Link |
|---|---|
| SFT LoRA | thegovind/azure-architect-qwen35-0.8b |
| SFT Merged | thegovind/azure-architect-qwen35-0.8b-merged |
| GRPO LoRA | thegovind/azure-architect-qwen35-0.8b-grpo |
| GRPO Merged (this) | thegovind/azure-architect-qwen35-0.8b-grpo-merged |
| Training Dataset | thegovind/azure-architecture-vqa |
| Benchmark | thegovind/azure-architecture-grpo-benchmark |