EauRouge-F1-Qwen3-8B

EauRouge-F1 - Qwen3-8B QLoRA adapter

LoRA adapter for unsloth/Qwen3-8B-unsloth-bnb-4bit, trained on the VineshF1/F1-Dataset F1 question/answer set: 5,872 training rows, paraphrases capped at 2 per fact plus 31 generated aggregate questions.

It answers short factual F1 questions about race results, qualifying, circuits and championship totals. It learns the answer format of the dataset; it does not learn new facts.

Usage

from unsloth import FastLanguageModel

model, tok = FastLanguageModel.from_pretrained(
    "VineshF1/EauRouge-F1-Qwen3-8B", max_seq_length=1024, load_in_4bit=True)
FastLanguageModel.for_inference(model)

prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Who won the 2021 Drivers' Championship?"}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False)
enc = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**enc, do_sample=False, max_new_tokens=200,
                     repetition_penalty=1.0, pad_token_id=tok.pad_token_id)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))

Keep enable_thinking=False and repetition_penalty=1.0 - seasons tokenize digit by digit, so higher penalties corrupt years.

Results

Measured with eval_adapter.py on a source-level split: no answer appears on both sides. The unseen-facts gap is inside the +/-10 pp noise for n=100.

Training

Eval loss: 0.7312 (step 50) -> 0.6997 (100) -> 0.7024 (150) -> 0.6998 (200) -> 0.7001 (216). Flat from step 100; the last 116 steps changed it by 0.0004.

Limitations

• Does not generalise to facts absent from training; it is a formatter, not a knowledge base.
• Trained on one-line answers, so open-ended questions get terse replies.

Evaluation

eval_adapter.py in https://github.com/VineshF1/EauRouge-F1 reproduces these numbers with one code path for adapter and base.

Author

Vinesh - github.com/VineshF1