Downloads · 30 days
0
RayTsai/chinese-grpo-qwen2.5-7b-40percent
chinese-grpo-qwen2.5-7b-40percent is a reinforcement learning model from RayTsai. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
使用GRPO (Group Relative Policy Optimization)方法訓練的中文大語言模型,專門優化處理敏感議題的中立性和推理能力。
Downloads · 30 days
0
Access
Public
Updated Jun 27, 2025
Repo size
51.8 MB
Likes
0
Public
Click a slice to open those files.
.safetensors40.4 MB · 72%
From the Hugging Face model README
使用GRPO (Group Relative Policy Optimization)方法訓練的中文大語言模型,專門優化處理敏感議題的中立性和推理能力。
LoRA配置:
r: 16
alpha: 32
dropout: 0.05
target_modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
訓練參數:
learning_rate: 3e-05
batch_size: 16
gradient_accumulation_steps: 2
num_epochs: 2
total_steps: 5,506
優化設置:
quantization: 4-bit
gradient_checkpointing: true
bf16: true
dataloader_num_workers: 0 # 解決pickle錯誤
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch
# 量化配置
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True
)
# 載入基礎模型
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen2.5-7B-Instruct",
quantization_config=quantization_config,
device_map="auto",
trust_remote_code=True
)
# 載入LoRA權重
model = PeftModel.from_pretrained(
base_model,
"RayTsai/chinese-grpo-qwen2.5-7b-40percent"
)
# 載入tokenizer
tokenizer = AutoTokenizer.from_pretrained("RayTsai/chinese-grpo-qwen2.5-7b-40percent")
# 使用模型
prompt = "問題:[您的問題]\n\n選項:\nA. [選項A]\nB. [選項B]\nC. [選項C]\nD. [選項D]\n\n請選擇正確答案並說明理由。"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
@misc{chinese-grpo-2025,
author = {Ray Tsai},
title = {Chinese GRPO Qwen2.5-7B 40% Dataset},
year = {2025},
publisher = {HuggingFace},
url = {https://huggingface.co/RayTsai/chinese-grpo-qwen2.5-7b-40percent}
}
Apache License 2.0
如有問題或合作意向,請通過HuggingFace平台聯繫。