gemma-3-12b-it-semeval2026-task9-polarization-lora

gemma-3-12b-it SALSA LoRA — multilingual polarization detection (SemEval-2026 Task 9, Subtask 1)

A LoRA adapter over unsloth/gemma-3-12b-it for binary polarization detection across 22 languages, trained for SemEval-2026 Task 9 Subtask 1.

The adapter is trained SALSA-style: rather than decoding free text, the model is scored by reading the logits of the two class tokens ("0" and "1") at a single <ANSWER> position, and the softmax over just those two logits is the class probability. This makes inference one forward pass per example and gives a calibrated probability you can threshold.

Usage

The prompt format matters — the adapter was trained on this exact string, and deviating from it (wrapping <ANSWER> in an assistant turn, or skipping the 800-character truncation) shifts the distribution away from what it saw.

import torch
from unsloth import FastLanguageModel

MAX_LENGTH, TEXT_MAX_LENGTH = 1024, 800

TASK_TEMPLATE = """Does the following text contain polarization?
A text is polarized if it incites division, hatred, or stereotyping towards other groups.

Answer in format:
<ANSWER>#Number</ANSWER>
where the number is one of the following:
0 - No
1 - Yes

The text:
<TEXT>
{text}
</TEXT>
"""

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="howarudo/gemma-3-12b-it-semeval2026-task9-polarization-lora",
    max_seq_length=MAX_LENGTH,
    dtype=None,
    load_in_4bit=False,
)
FastLanguageModel.for_inference(model)

def polarization_prob(text: str) -> float:
    messages = [{"role": "user", "content": TASK_TEMPLATE.format(text=str(text)[:TEXT_MAX_LENGTH])}]
    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + "<ANSWER>"
    enc = tokenizer(prompt, return_tensors="pt", truncation=True,
                    max_length=MAX_LENGTH, add_special_tokens=False).to(model.device)

    class_ids = torch.tensor([tokenizer.encode(c, add_special_tokens=False)[0] for c in ["0", "1"]],
                             device=model.device)
    # Autocast is required: the Gemma3 attention path computes in fp32 and
    # projects through bf16 weights.
    with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
        logits = model(**enc).logits[0, -1, :]
    return torch.softmax(logits.index_select(0, class_ids).float(), dim=-1)[1].item()

print(polarization_prob("Your text here."))   # P(polarized)

For batched scoring, right-pad the sequences and read the logits at each row's own last non-padding position — padding sits after the answer token, so under causal attention it cannot influence the class logits.

License

Governed by the Gemma Terms of Use, inherited from the base model.