mistral-7b-arc-qlora-mc

Mistral 7B ARC QLoRA with multiple-choice loss

This is the validation-selected checkpoint from an ARC multiple-choice (MC) fine-tuning experiment starting from Mistral-7B-v0.1. It is a task-specific model, not an instruction-tuned chat assistant. The root contains the complete FP16 model and tokenizer.

Recorded evaluation

All three conditions below used the same 1,172 ARC-Challenge test questions, 25-shot context, FP16, batch size 4, SDPA, no chat template and no added BOS/EOS. The two fine-tuned models were evaluated after merging their original saved adapters into a fresh, unquantized FP16 base.

MC improves acc_norm by 19.97 percentage points over the base and 14.16 points over the SFT reference.

Training and selection

Training used the same fixed, de-duplicated set of 3,353 ARC-Challenge and ARC-Easy training questions as the SFT reference, retaining all original answer choices. Training questions whose normalized text matched held-out questions were excluded. Held-out answer labels were not used for training. This checks the specified text overlap rule, not all possible semantic overlap or the base model's pretraining contamination.

For each question, the model scores each answer completion after Question: {question}\nAnswer:. Its score is the sum of answer-token log probabilities divided by Python len(original_choice_text), including original spaces but excluding the added separator space. Cross-entropy over these character-normalized choice scores (temperature 1) is averaged per question. Only answer-token log probabilities contribute to each choice score; prompt and padding positions are excluded from the sum. The softmax over choice scores is a relative distribution, not calibrated confidence. QLoRA used NF4 with double quantization and BF16 computation; LoRA rank 8, alpha 16, dropout 0.05, applied to q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj and down_proj.

Training stopped early at epoch 11. Epoch 1 / step 210 was selected using the highest ARC-Challenge validation normalized accuracy: 237 / 299 (79.26%), evaluated with NF4/BF16. Ties retain the earlier checkpoint. Selection was frozen before the final test.

Revisions and representation

• Base and tokenizer: mistralai/Mistral-7B-v0.1, revision 27d67f1b5f57dc0953326b2601d68371d40ea8da.
• Dataset: allenai/ai2_arc, revision 210d026faf9955653af8916fad021475a3f00453.

The published FP16 weights were produced by merging the selected LoRA adapter into the original, unquantized FP16 base model. The tokenizer vocabulary is unchanged; saved settings disable added BOS/EOS, use <unk> (ID 0) as PAD and right padding.

Loading the complete model

Install the versions in requirements.txt.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "InkeeJung/mistral-7b-arc-qlora-mc"
tokenizer = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(
    REPO,
    torch_dtype=torch.float16,
    device_map={"": 0},
    attn_implementation="sdpa",
    use_safetensors=True,
).eval()

Limits and license

This is one task-focused run with one training seed. Validation and test scores differ in both data split and numerical representation, so their difference cannot be attributed solely to quantization. Benchmark specialization and pretraining contamination remain possible.

The base was released under Apache 2.0. This derivative retains that license and attributes the base to Mistral AI. The derivative changes consist of the ARC MC-loss fine-tuning described above.