xdomain-ser-ranker

xdomain-ser-ranker

A LoRA adapter for Llama-3.2-3B-Instruct that scores candidate meaning representations in the over-generate-and-rank (OGR) stage of our cross-domain SER evaluation framework (GEM 2026). Companion to DavanHarrison/xdomain-ser-extractor.

What it does

The extractor produces k candidate MRs via beam search. This ranker assigns each candidate an integer grade on a 7-point rubric (0-6). Probability-weighted aggregation over the grade tokens selects the best candidate. Adding OGR raises extraction F1 from 0.819 to 0.918 (k=10) and SER agreement from 0.752 to 0.88.

Usage

This is a PEFT adapter. The base model must be obtained separately from Meta and is governed by the Llama 3.2 Community License (see License below).

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")
ranker = PeftModel.from_pretrained(base, "DavanHarrison/xdomain-ser-ranker")
tokenizer = AutoTokenizer.from_pretrained("DavanHarrison/xdomain-ser-ranker")

The ranker is meant to be used after the extractor. See the code repository for the end-to-end pipeline: https://github.com/Vrindiesel/xdomain-ser

Training

• Base model: meta-llama/Llama-3.2-3B-Instruct
• Method: 4-bit QLoRA supervised fine-tuning
• LoRA config: r=4, alpha=16, dropout=0.05, rank-stabilized (rsLoRA), EVA initialization
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Training data: contrastive negatives generated by perturbing gold MRs from multi_ser_v9; ranker learns to grade candidate MRs 0-6 against the gold

Intended use

Candidate selection within the SER extraction pipeline. Research use. English-only.

Limitations

The ranker is trained and validated only as part of the extraction pipeline on task-oriented domains, English-only. Standalone use outside that pipeline is not characterized.

License

The adapter weights and accompanying code are released under the Apache License 2.0, covering our contribution only.

Use of this adapter requires the base model, meta-llama/Llama-3.2-3B-Instruct, distributed under the Llama 3.2 Community License Agreement. You must obtain the base model from Meta and accept that license to use this adapter. This release does not redistribute any Meta weights.

Citation

@inproceedings{harrison2026xdomainser,
  title     = {Cross-Domain Semantic Fidelity Evaluation for Meaning-to-Text NLG},
  author    = {Harrison, Davan and Walker, Marilyn},
  booktitle = {Proceedings of the GEM Workshop at ACL 2026},
  year      = {2026}
}