NYCU-IAlI-DL2026-LLM3-GRPO

ruby2022/NYCU-IAlI-DL2026-LLM3-GRPO

NYCU-IAlI-DL2026-LLM3-GRPO is a reinforcement learning model from ruby2022. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.

This repository contains: - LoRA adapter fine-tuned from Qwen/Qwen3-14B via GRPO reinforcement learning - Training data (train-reasoning-v2.csv) used for GRPO training

Downloads · 30 days

20% of all-time downloads

All-time downloads

Public

Repo size

268 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors257 MB · 94%

At a glance

Task: Reinforcement Learning
Library: peft
License: apache-2.0
Access: Public
Created: Jun 20, 2026
Updated: Jun 20, 2026
SHA: 7e91a0a7

Base models

Qwen/Qwen3-14B

Model card

From the Hugging Face model README

NYCU IAI-DL 2026 LLM3 GRPO — Qwen3-14B LoRA Adapter

This repository contains:

LoRA adapter fine-tuned from Qwen/Qwen3-14B via GRPO reinforcement learning
Training data (train-reasoning-v2.csv) used for GRPO training

📊 Results

Submission	Inference	Public	Private
Qwen3-14B + GRPO	Majority Vote N=8	0.694	0.698
Qwen3-14B + GRPO	Greedy (N=1)	0.687	0.685

🏆 Competition

Kaggle: nycu-i-al-i-dl-2026-llm-3-grpo

Task: Fine-tune a Chinese-released LLM using GRPO so that it correctly answers Traditional Chinese single-choice questions (A/B/C/D).

🔧 LoRA Configuration

Parameter	Value
Base model	`Qwen/Qwen3-14B`
Rank (r)	16
Alpha	16
Dropout	0.0
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable params	~1.5%
Training method	GRPO (Group Relative Policy Optimization)

📁 Files

File	Description
`adapter_config.json`	LoRA adapter configuration
`adapter_model.safetensors`	LoRA adapter weights (~245 MB)
`tokenizer.json`	Tokenizer
`tokenizer_config.json`	Tokenizer configuration
`chat_template.jinja`	Qwen3 chat template with thinking enabled
`train-reasoning-v2.csv`	Training dataset (3,199 samples)
`kaggle_test_set_792.csv`	Test set (792 questions, no labels)

🚀 Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model_name = "Qwen/Qwen3-14B"
adapter_name = "ruby2022/NYCU-IAlI-DL2026-LLM3-GRPO"

tokenizer = AutoTokenizer.from_pretrained(adapter_name)
model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_name)
model = model.merge_and_unload()

🏗️ Training Details

Algorithm: GRPO (Group Relative Policy Optimization)
Key innovation: Option Shuffling augmentation to prevent reward collapse
Reward functions: correctness_reward (+2.0/−1.0) + format_reward (+0.5)
Inference: Majority Voting (N=8, temperature=0.6)
Thinking mode: Qwen3 native <think>...</think> enabled throughout

See the GitHub repository for full source code, training scripts, and logs.

⚙️ Hardware

GPU: NVIDIA RTX PRO 6000 Blackwell (96 GB VRAM)
Training: Single GPU, ~500 steps GRPO

Task: Reinforcement Learning
Library: peft
License: apache-2.0
Languages: zh
Created: Jun 20, 2026
Updated: Jun 20, 2026