qwen3-4b-physics-recontext

Qwen3-4B — Concept-Preserving Physics Re-Contextualizer (v1, LoRA adapter)

A QLoRA fine-tune of Qwen3-4B that takes a high-school physics flashcard (question + answer) and writes one new question testing the same underlying principle in a clearly different real-world scenario — a far transfer, reliably, without paraphrasing, concept drift, or added knowledge. The model is not told the principle; it infers and preserves it.

This repo is the LoRA adapter only — load it on top of the base unsloth/Qwen3-4B.

Behavior spec (the target)

Given a flashcard (Q + A, no principle label), output one new question in a distinctly different real-world context whose answer relies on the same physics principle and only that principle, adds no new required concepts, and preserves difficulty — never a reworded original, never a different principle.

Results (base vs tuned)

126 held-out flashcards, each model sees only Q+A. A variant passes only if a cross-family LLM judge passes all 5 rubric dimensions AND it is not a lexical near-copy of the original (similarity < 0.60).

Δ = +34 points. Largest gains on context_novelty (0.52 → 0.74) — i.e. the tuned model relabels far less and genuinely re-contextualizes. Full report and per-dimension / per-topic breakdowns are in the project repo.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "unsloth/Qwen3-4B"
ADAPTER = "meric533/qwen3-4b-physics-recontext"

tok = AutoTokenizer.from_pretrained(ADAPTER)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)

SYSTEM = (
    "You rewrite physics flashcards to build transfer of learning. Given a flashcard,\n"
    "output ONE new question that tests the exact same underlying physics principle as\n"
    "the flashcard, but set in a clearly different real-world scenario. Do not reword\n"
    "the original scenario, do not require any additional concept or formula, and keep\n"
    "the difficulty the same. Output only the new question, with no preamble."
)
user = (
    "Flashcard:\nQ: What causes atmospheric pressure?\n"
    "A: The weight of the atmosphere pressing down due to gravity.\n\n"
    "Write one new question testing the same underlying concept in a different real-world scenario."
)
msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": user}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0, ids.shape[-1]:], skip_special_tokens=True))

Training

• Method: 4-bit QLoRA (Unsloth), response-only SFT (trains on the assistant turn).
• LoRA: r=16, α=16, targets = all attention + MLP projections.
• Hyperparameters: 3 epochs, effective batch 16, lr 2e-4, max_seq_len 1024.
• Data: meric533/physics-recontext-sft (967 train / 107 val, chat-format far-transfer pairs).

Intended use & limitations

• Intended: research / educational tooling for generating transfer-of-learning practice questions from physics flashcards. Non-commercial (see license).
• Limitations: the tuned model's creativity occasionally introduces a false premise or contradiction (the "invalid" failure mode, ~15% of outputs). Trivial template flashcards (e.g. "which has more thermal energy?") admit only relabeling. Not an authoritative physics source; verify outputs before classroom use.

License

CC-BY-NC-SA 4.0. The adapter derives from Qwen3-4B (Apache-2.0) but was trained on data derived in part from non-commercial sources (SciQ, ScienceQA), so the adapter is released under the more restrictive NC-SA terms.