qwen3-4b-merged-preference-classifier

Model Card for Model ID

This model is a fine-tuned version of Qwen/Qwen3-4B-Instruct-2507 for text classification, specifically designed to predict user preferences between two chatbot responses given a prompt. It was trained on the Kaggle LLM Classification Finetuning dataset, where the task is to select which response (A or B) is better, or if both are equally good (Tie). The model outputs one of three classes:

• Winner A
• Winner B
• Tie

Model Details

To use this model i recommend to use the prompt that i used during training and format your input as follows:

You are an expert AI assistant that is specialized in selecting user preferences.
The task it that you are provided with a prompt and two responses (A and B) to that prompt from different LLMs.
The possible outcomes are 3 classes:
- Winner A: Response A is better
- Winner B: Response B is better
- Tie: Both responses are equally good

The prompt to the models is:
{prompt}

Then response A is:
{response_a}

And response B is:
{response_b}

Based on the above, classify which response is better by choosing one of the following options: "Winner A", "Winner B", or "Tie".

Usage:

from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
model = AutoModelForSequenceClassification.from_pretrained(
    "Vmpletsos/qwen3-4b-merged-preference-classifier",
    num_labels=3,
    torch_dtype=torch.bfloat16,
    device_map="cuda"
)

Make sure to set the num labels to 3 as the model was trained as a 3 classes classification problem