llama-3.2-3b-it-grpo-250404-GGUF

llama-3.2-3b-it-grpo-250404-GGUF

ReZero-v0.1-llama-3.2-3b-it-grpo-250404 is a research project focused on enhancing the search abilities of small language models by training them to develop robust search strategies rather than memorizing static data. The model, built on a Llama-3.2-3B backbone, interacts with multiple synthetic search engines that each have unique retrieval mechanisms, enabling it to refine queries iteratively and persist in finding exact answers using reinforcement learning. The repository provides setup instructions, including environment configuration and dependency installation, as well as scripts to train the model or regenerate synthetic training data. Demonstrations can be run through a Gradio interface, and the release includes comprehensive experiment logs on reward strategies and search quality. The model and associated resources are open-source and accessible to the research community, with further details on experiments and references provided in the documentation.

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):