EvoLM-8B-Rubric-GGUF

EvoLM-8B-Rubric GGUF

This repository provides GGUF quantizations for stellalisy/EvoLM-8B-Rubric.

EvoLM-8B-Rubric is the rubric-generator checkpoint from the paper EvoLM: Self-Evolving Language Models through Co-Evolved Discriminative Rubrics. It represents the final rubric checkpoint at global training step 1000 (rubric/step_1000) fine-tuned from Qwen/Qwen3-8B on the allenai/tulu-3-sft-mixture.

Rather than producing scalar rewards directly, this model dynamically generates question-specific, weighted evaluation rubrics with explicit scoring criteria to guide subsequent evaluation or reward judging.

All GGUF files were converted from the original safetensors weights using llama.cpp at native BF16 precision, then quantized into standard k-quant variants.

---

Available Files and Quantizations

---

Rubric Generation Role & Format

EvoLM-8B-Rubric generates weighted, multi-criteria evaluation rubrics tailored to input prompts. In the paper's pipeline, these generated criteria are consumed by a downstream judge model (such as Qwen3-1.7B) to evaluate candidate responses across distinct quality dimensions.

Prompt Format

The checkpoint uses standard Qwen ChatML formatting and supports <think> reasoning blocks prior to emitting the structured rubric:

<|im_start|>system
You are an expert evaluation assistant. Given a user query or task, generate a structured, weighted rubric containing 2 to 5 atomic criteria to evaluate candidate responses.<|im_end|>
<|im_start|>user
Write a short explanation of quantum entanglement for high school students.<|im_end|>
<|im_start|>assistant
<think>
Analyze task requirements: conceptual accuracy, age-appropriate analogy, clarity, engagement.
</think>

---

How to Use

1. With llama.cpp

Run interactive generation with llama-cli:

llama-cli -m ./EvoLM-8B-Rubric-Q4_K_M.gguf \
    -p "<|im_start|>system\nYou are an evaluation assistant that creates weighted rubrics.<|im_end|>\n<|im_start|>user\nExplain why the sky is blue.<|im_end|>\n<|im_start|>assistant\n<think>\n" \
    -n 512 \
    -c 4096 \
    --temp 0.6

Launch a local OpenAI-compatible API server:

llama-server -m ./EvoLM-8B-Rubric-Q4_K_M.gguf \
    --host 127.0.0.1 \
    --port 8080 \
    -c 4096

2. With Ollama

Create a file named Modelfile in the same directory:

FROM ./EvoLM-8B-Rubric-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
{{- if .Response }}
{{ .Response }}
{{- else }}
<think>
{{- end }}
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.6

Build and run the model:

ollama create evolm-8b-rubric -f Modelfile
ollama run evolm-8b-rubric

3. With LM Studio

1. Place the desired .gguf file into your local LM Studio models directory (e.g. ~/.cache/lm-studio/models/stellalisy/EvoLM-8B-Rubric-GGUF).
2. Select EvoLM-8B-Rubric in the model loader tab.
3. Select the ChatML preset as the prompt template.

---

Model Architecture Details