qwen3-14b-stem-qa

Qwen3-14B STEM Q&A (LoRA Adapter)

This is a LoRA adapter fine-tuned on Qwen3-14B for STEM Q&A tasks.

Author: Khadim Hussain · Project write-up: STEM reasoning LLMs

For the merged full model (larger download, standalone use), see: khadim-hussain/qwen3-14b-stem-qa-merged

Model Description

• Base Model: Qwen/Qwen3-14B
• Type: LoRA Adapter (QLoRA 4-bit)
• Training: Fine-tuned on STEM Q&A dataset

Usage

from unsloth import FastLanguageModel

# Load model with adapter
model, tokenizer = FastLanguageModel.from_pretrained(
    "khadim-hussain/qwen3-14b-stem-qa",
    max_seq_length=2048,
    load_in_4bit=True,
)

prompt = "<|im_start|>user\nWhat is DNA?<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Configuration

• Method: QLoRA (4-bit quantization)
• Optimizer: AdamW 8-bit
• Learning Rate: 1e-4
• Batch Size: 1 (with gradient accumulation 16)
• Framework: Unsloth + TRL

GGUF Version

GGUF quantized versions are available for use with Ollama/llama.cpp:

• See: khadim-hussain/qwen3-14b-stem-qa-gguf

Running the adapter in llama.cpp

Ollama 0.34.1 and later no longer load LoRA adapters, so in Ollama use the merged GGUF above. To keep the adapter separate, convert it with llama.cpp's convert_lora_to_gguf.py (pass --base-model-id Qwen/Qwen3-14B, because the adapter config points at Unsloth's 4-bit training copy) and load it with llama-server --lora on top of a Qwen3-14B GGUF. Tested steps and numbers: Running a LoRA adapter after Ollama removed support.

Acknowledgments

• Qwen Team (Alibaba) - Qwen3-14B base model
• Unsloth - Fast fine-tuning framework
• Hugging Face - TRL, PEFT, Transformers

Citation

If you use this model, please cite:

@misc{hussain2026qwen3-stem,
  author = {Hussain, Khadim},
  title = {Qwen3-14B STEM Q&A: Fine-tuned for Science Education},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/khadim-hussain/qwen3-14b-stem-qa}
}

License

Apache 2.0 (inherited from Qwen3)