Downloads · 30 days
10
12% of all-time downloads
yavuz-ai/qwen2.5-3b-grpo-finqa
qwen2.5-3b-grpo-finqa is a text generation model from yavuz-ai. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
A LoRA policy from GRPO + RLVR on FinQA (SEC-filing financial reasoning with checkable numeric answers). The reward is verifiable ground truth - is the final number correct, after normalizing percent / scale / sign -…
Downloads · 30 days
10
12% of all-time downloads
All-time downloads
85
Public
Repo size
131 MB
Likes
0
Public
Click a slice to open those files.
.safetensors120 MB · 91%
From the Hugging Face model README
A LoRA policy from GRPO + RLVR on FinQA (SEC-filing financial reasoning with checkable numeric
answers). The reward is verifiable ground truth - is the final number correct, after normalizing
percent / scale / sign - so there is no reward model and no value model (beta=0, group-average
baseline). The online arm of finqa-rlvr.
FinQA validation accuracy (greedy, n=200): 0.530 -> 0.525 (-1 question). Flat, and within noise at
this eval size. The honest finding from the project is that at this scale method choice is
second-order: the reward design and the base-model headroom dominate, not online-vs-offline. GRPO
here ran at a conservative lr=1e-6 that decayed to ~0, so the flatness is partly an
optimization-budget artifact - noted, not chased. See the repo for the full teardown and the offline
(DPO) counterpart trained on the same signal.
GRPO (TRL, use_vllm colocate), LoRA r=16 / alpha=32, group size 8, beta 0, correctness + format
reward, 600 steps, 1x A100.
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct")
model = PeftModel.from_pretrained(base, "yavuz-ai/qwen2.5-3b-grpo-finqa")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")