Downloads · 30 days
19
18% of all-time downloads
eagle0504/multireward-grpo-fintech-single-qwen2.5-1.5b
multireward-grpo-fintech-single-qwen2.5-1.5b is a text generation model from eagle0504. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter trained with SINGLE advantage formulation from "Conditioned Multi-Reward Advantage Estimation: A Finite-Sample Analysis".
Downloads · 30 days
19
18% of all-time downloads
All-time downloads
103
Public
Repo size
29.3 MB
Likes
0
Public
Click a slice to open those files.
.safetensors17.5 MB · 59%
From the Hugging Face model README
LoRA adapter trained with SINGLE advantage formulation from "Conditioned Multi-Reward Advantage Estimation: A Finite-Sample Analysis".
This is the difference vs the AN baseline:
| AN (baseline) | NA (paper's recommendation) | |
|---|---|---|
| Step 1 | aggregate channels: s_j = w^T r_j | per-channel normalize: z_jl = (r_jl - mean_l) / std_l |
| Step 2 | group-normalize: A_j = (s_j - mean) / std | aggregate: A_j = sum_l w_l z_jl |
| Influence | dominated by high-σ channel (Prop 1) | weight-proportional (Prop 1) |
| MSE floor | sensitive to scalarization | (τ²/m) w^T C w (Thm 3) |
huggingface.co/datasets/eagle0504/multireward-grpo-fintech-customer-comms
— synthetic fintech customer-service conversations, 300 scenarios, with
reward channels:
compliance (binary): the harder gatepoliteness_gated (continuous): gated by complianceaction (binary): clear next-step indicatorfrom peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "Qwen/Qwen2.5-1.5B-Instruct"
adapter = "eagle0504/multireward-grpo-fintech-single-qwen2.5-1.5b"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16")
model = PeftModel.from_pretrained(model, adapter)
model.eval()
# ... generate ...
See metrics.json in the repo for the full training trajectory (loss,
PG loss, KL, per-step reward).
@misc{yin2026multireward,
title={Conditioned Multi-Reward Advantage Estimation: A Finite-Sample Analysis},
author={Yin, Yiqiao},
year={2026},
}
Apache-2.0 (matches the Qwen base model).