Qwen2.5-7B-GrapheneContact-QDoRA-CoT

Qwen2.5-7B-GrapheneContact-QDoRA-CoT

A domain-specific language model fine-tuned for condensed matter physics, specifically graphene–metal contact resistance research. This 7B model, trained on only 64 question–answer pairs, outperforms a 235B-parameter model with open-book access under closed-book evaluation conditions.

Key Result

A fine-tuned 7B model with no access to the source paper outperforms a 33× larger model that receives the paper's content as a prompt.

Model Details

Model Description

• Developed by: Junsu Park
• Affiliation: Department of Physics and Photon Science, Gwangju Institute of Science and Technology (GIST), Korea
• Base model: Qwen/Qwen2.5-7B-Instruct
• Fine-tuning method: QDoRA (Quantized Weight-Decomposed Low-Rank Adaptation) + Chain-of-Thought reasoning
• Language: English
• License: Apache 2.0
• Domain: Condensed matter physics — graphene–metal contact resistance (TLCD/TECD devices)

Model Sources

• Repository: GitHub (coming soon)
• Research Article: "Small Data, Small Model, Superior Domain Accuracy: Fine-Tuning a 7B Language Model as a Condensed Matter Physics Expert with 64 Examples"(https://www.preprints.org/manuscript/202603.1691)

Uses

Direct Use

This model is designed to answer domain-specific questions about graphene–metal contact resistance, including device structure (TLCD/TECD), fabrication processes, theoretical models (Landauer, BTH), and experimental results. It serves as a personalized research assistant for this specific domain.

Out-of-Scope Use

This model is trained on a single research paper and should not be used for general scientific reasoning, other physics domains, or any task outside graphene–metal contact resistance research.

Training Details

Training Data

64 expert-crafted question–answer pairs from a single condensed matter physics paper, organized into five categories:

Each answer includes Chain-of-Thought (CoT) reasoning with <think> and <final> tags.

Training Hyperparameters

Hardware

Evaluation

30 expert-designed questions (distinct from training set) evaluated by the domain expert (first author of the source paper) using a three-tier rubric: ✓ (correct), △ (partial), ✗ (wrong).

Results by Category

Error Pattern Analysis

Each model exhibits qualitatively distinct failure modes:

• Base 7B: Dominated by partial coverage (PC) — generates relevant but insufficiently specific responses
• 235B: Dominated by over-extrapolation (OE) — fills knowledge gaps with plausible but incorrect scientific details
• FT 7B (this model): Occasional directional reversal (DR) — correctly identifies relevant concepts but sometimes inverts their relationships

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
    "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT"
)

# Load fine-tuned adapter
model = PeftModel.from_pretrained(
    base_model,
    "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT"
)

# Inference
messages = [
    {"role": "user", "content": "Why does increasing contact area reduce contact resistance in graphene-metal interfaces?"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Citation

@article{park2026small,
  title={Small Data, Small Model, Superior Domain Accuracy: Fine-Tuning a 7B Language Model as a Condensed Matter Physics Expert with 64 Examples},
  author={Park, Junsu},
  year={2026},
  institution={Gwangju Institute of Science and Technology (GIST)}
}

Contact

• Author: Junsu Park
• Affiliation: GIST, Department of Physics and Photon Science
• Hugging Face: pjspjs0987

Framework versions

• PEFT 0.18.0