qwen3-4b-k12kgraph

Qwen3-4B-K12KGraph

Qwen3-4B-K12KGraph is a full-parameter supervised fine-tuning (SFT) model based on Qwen/Qwen3-4B-Base. It was trained on the K12-Train data associated with the K12-KGraph project for educational question answering.

This repository contains a standalone Transformers checkpoint. The original Qwen3-4B-Base weights do not need to be downloaded separately.

Model details

• Architecture: Qwen3ForCausalLM
• Parameters: approximately 4B
• Fine-tuning method: full-parameter SFT
• Training framework: LLaMA-Factory
• Context length used for preprocessing: 4,096 tokens
• Weight dtype: bfloat16
• Languages: Chinese and English
• License: Apache License 2.0

Intended use

The model is intended for research on K-12 educational question answering, knowledge-intensive reasoning, and evaluation of models trained with knowledge-graph-derived educational data.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "lhpku20010120/qwen3-4b-k12kgraph"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "Explain the difference between the light-dependent and light-independent reactions in photosynthesis.",
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        inputs,
        max_new_tokens=512,
        do_sample=False,
    )

answer = tokenizer.decode(
    outputs[0, inputs.shape[-1]:],
    skip_special_tokens=True,
)
print(answer)

The checkpoint uses the included chat_template.jinja, whose dialogue format is based on Human: and Assistant: markers.

Training

The following information was recovered from the exported trainer metadata:

Limitations

• The model may reproduce errors and biases present in the base model or fine-tuning data.
• Generated answers may be incorrect, incomplete, or hallucinated. The model should not be treated as an authoritative source for high-stakes educational decisions.
• Its strongest expected domain is K-12 educational content; performance may degrade outside that domain.

Data and copyright

Users are responsible for complying with the licenses and terms associated with the base model, training data, benchmark data, and generated content. The release of model weights does not grant rights to reproduce any third-party textbook or benchmark material.

Acknowledgements

This model is based on Qwen3-4B-Base and was trained with LLaMA-Factory. Please cite the Qwen3 and K12-KGraph projects when using this checkpoint in research.