GRAM-LLaMA3.2-3B-RewardModel-GGUF

GRAM-LLaMA3.2-3B-RewardModel-GGUF

GRAM-LLaMA3.2-3B-RewardModel is a generative reward model fine-tuned from the Llama-3.2-3B-Instruct base model released by NiuTrans. It is designed to improve reward generalization for large language models (LLMs) by leveraging a novel training approach that first pre-trains on large unlabeled datasets and then fine-tunes using supervised labeled data. The training uses label smoothing and optimizes a regularized ranking loss, bridging generative and discriminative reward modeling techniques. This enables the model to be applied flexibly across a variety of tasks without the usual need for extensive fine-tuning on task-specific datasets.

GRAM-LLaMA3.2-3B-RewardModel is evaluated on the JudgeBench benchmark, which covers domains such as Chat, Code, Math, and Safety. It achieves a competitive average score of 69.9 across these categories, demonstrating strong capability for use as an open-source plug-and-play reward model that can align LLMs effectively without retraining reward models from scratch. The repository includes usage examples that let users directly apply this reward model for assessing and ranking the quality of AI-generated responses in an impartial manner.

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):