aisa-arabicfc-ens-v3

AISA-ArabicFC (ArabicNLP 2026) - Team Yalla, system ens_v3

Artifacts needed to reproduce the two submitted files under verification: TEST_A_ens_v3.jsonl and TEST_B_ens_v3.jsonl.

The submitted system is a per-argument majority vote over three member models. Each member is run separately over the 1,125 test prompts, then the three prediction files are combined. No model here is used on its own.

Contents

Each adapters/* folder holds adapter_model.safetensors plus adapter_config.json and the tokenizer files that were saved alongside it. merged/argfocus_full_merged is a standard sharded Hugging Face model (model-0000X-of-00004.safetensors plus model.safetensors.index.json).

Two things to know before loading

1. grpo_aff must be loaded on the merged model, not on a public base. It was trained after a supervised LoRA stage was merged permanently into the weights, so its delta is only meaningful relative to merged/argfocus_full_merged. Loading it on public Qwen/Qwen2.5-7B-Instruct will not reproduce our predictions.

2. adapters/grpo_aff/adapter_config.json records base_model_name_or_path = runs/argfocus_full_merged. That is a leftover local path from our training machine and it will not exist on yours. Pass the base explicitly instead of relying on that field:

base = AutoModelForCausalLM.from_pretrained("<local>/merged/argfocus_full_merged")
model = PeftModel.from_pretrained(base, "<local>/adapters/grpo_aff")

The inference script we supplied (run_ens_v3.sh) already passes --base explicitly, so the field is never used there.

Reproducing

Use the accompanying archive aisa_VERIFY_ens_v3.zip, which contains the code, run instructions, pinned dependency versions, and the submitted files with checksums.

huggingface-cli download mojo28/aisa-arabicfc-ens-v3 --local-dir ./models
MODELS=./models ./run_ens_v3.sh

Re-running is not bit-identical, because batched GPU inference is not bit-deterministic even with greedy decoding. Across three independent runs we measured 99.20% to 99.38% row agreement with the submitted file, while independent runs agreed with each other by the same margin (98.93% to 99.38%). Tool selection matched 100% every time. Details and per-run outputs are in the archive.