SWaP-MT-Phi-4-mini-instruct-ENVI-GRPO-adapter

SWaP-MT EN→VI Adapter (GRPO)

This repository provides a PEFT (LoRA) adapter for English → Vietnamese translation trained with the SWaP-MT pipeline: SFT → COMET/chrF preference mining → DPO → GRPO (mixed rewards).

Base model

• microsoft/Phi-4-mini-instruct

Metrics (reported from this run)

PhoMT (dev)

• SFT : BLEU 27.61, chrF 47.36
• DPO : BLEU 28.29, chrF 48.21
• GRPO : BLEU 28.22, chrF 48.41

IWSLT2015 en-vi

• GRPO valid: BLEU 29.40, chrF 48.99 (n=1268)
• GRPO test : BLEU 29.40, chrF 48.99 (n=1268)

Files

This repo contains:

• adapter_config.json + adapter_model.safetensors (the adapter)
• Tokenizer files for reproducible decoding
• swapmt_cfg.json and swapmt_metrics.json

How to use

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

base_id = "microsoft/Phi-4-mini-instruct"
adapter_id = "Kiffaz11/SWaP-MT-Phi-4-mini-instruct-ENVI-GRPO-adapter"

tok = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)  # use tokenizer shipped with adapter
if tok.pad_token_id is None:
    tok.pad_token = tok.unk_token if tok.unk_token_id is not None else tok.eos_token

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.float16,
)

base = AutoModelForCausalLM.from_pretrained(
    base_id,
    trust_remote_code=True,
    device_map="auto",
    quantization_config=bnb,
    torch_dtype=torch.float16,
    attn_implementation="sdpa",
)

model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

SYSTEM_PROMPT = "You are a high-quality translation system. Translate English to Vietnamese. Output ONLY the Vietnamese translation."
def make_prompt(en):
    msgs = [
        {"role":"system","content":SYSTEM_PROMPT},
        {"role":"user","content":f"English:\n{en}\n\nVietnamese:"},
    ]
    return tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)

prompt = make_prompt("This is a test sentence.")
inputs = tok([prompt], return_tensors="pt").to(model.device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=96, num_beams=4)

print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).split("\n")[0].strip())