Downloads · 30 days
0
KhushRai78/qwen3-0.6B-GRPO-LORA
qwen3-0.6B-GRPO-LORA is a machine learning model from KhushRai78. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This model is a fine-tuned version of Qwen/Qwen3-0.6B trained on a subset of the MS MARCO dataset for improved question-answering and information retrieval capabilities.
Downloads · 30 days
0
Access
Public
Updated Jun 9, 2025
Repo size
133 MB
Likes
1
Public
Click a slice to open those files.
.pt81.1 MB · 59%
From the Hugging Face model README
This model is a fine-tuned version of Qwen/Qwen3-0.6B trained on a subset of the MS MARCO dataset for improved question-answering and information retrieval capabilities.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
# Model configuration
base_model_id = "Qwen/Qwen3-0.6B"
trained_model_path = "KhushRai78/qwen3-0.6B-GRPO-LORA" # Replace with your model path
optimal_dtype = torch.float16 # Adjust as needed
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(
trained_model_path,
trust_remote_code=True,
padding_side="left"
)
# Load base model and resize embeddings to match trained tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype=optimal_dtype,
device_map="auto",
trust_remote_code=True,
low_cpu_mem_usage=True
)
# Resize embeddings to match the trained tokenizer
base_model.resize_token_embeddings(len(tokenizer))
# Load trained model (LoRA)
trained_model = PeftModel.from_pretrained(base_model, trained_model_path)
# Generate text
prompt = "Question: Who did Rollo sign the treaty of Saint-Clair-sur-Epte with? Context: In the course of the 10th century, the initially destructive incursions of Norse war bands into the rivers of France evolved into more permanent encampments that included local women and personal property. The Duchy of Normandy, which began in 911 as a fiefdom, was established by the treaty of Saint-Clair-sur-Epte between King Charles III of West Francia and the famed Viking ruler Rollo, and was situated in the former Frankish kingdom of Neustria. The treaty offered Rollo and his men the French lands between the river Epte and the Atlantic coast in exchange for their protection against further Viking incursions. The area corresponded to the northern part of present-day Upper Normandy down to the river Seine, but the Duchy would eventually extend west beyond the Seine. The territory was roughly equivalent to the old province of Rouen, and reproduced the Roman administrative structure of Gallia Lugdunensis II (part of the former Gallia Lugdunensis)."
messages = [
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
enable_thinking=False # Switches between thinking and non-thinking modes. Default is True.
)
model_inputs = tokenizer([text], return_tensors="pt").to(trained_model.device)
# conduct text completion
generated_ids = trained_model.generate(
**model_inputs,
max_new_tokens=100
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
# parsing thinking content
try:
# rindex finding 151668 (</think>)
index = len(output_ids) - output_ids[::-1].index(151668)
except ValueError:
index = 0
thinking_content = tokenizer.decode(output_ids[:index], skip_special_tokens=True).strip("\n")
content = tokenizer.decode(output_ids[index:], skip_special_tokens=True).strip("\n")
print("thinking content:", thinking_content)
print("response:", content)
Dataset: SQuAD v2.0 (1,003 answerable test cases)
| Model | ROUGE | Exact Match (EM) | F1 Score |
|---|---|---|---|
| Original Qwen3-0.6B | 0.181 | 0.055 | 0.173 |
| Fine-tuned Model | 0.404 | 0.141 | 0.391 |
The fine-tuned model demonstrates significant improvements across all metrics, particularly excelling in contextual understanding and answer generation quality.
If you use this model, please cite the original Qwen3 work:
@misc{qwen3technicalreport,
title={Qwen3 Technical Report},
author={Qwen Team},
year={2025},
eprint={2505.09388},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.09388},
}
You may also reference this fine-tuned version:
@misc{qwen3-0.6-GRPO-LORA,
title={Qwen3-0.6B Fine-tuned on MS MARCO},
author={[Khushwant Rai]},
year={2025},
note={Fine-tuned version of Qwen3-0.6B using LoRA on MS MARCO dataset}
}
Model card created following standard ML model documentation practices. Last updated: June 2025