new4b

Susu11/new4b

QLoRA adapters for a Grade 10 Socratic science tutor on Qwen/Qwen3-4B-Instruct-2507.

This Hub repo is Qwen-only. Phi-3 adapters live in a separate model repo (HF_HUB_REPO). GitHub Sushey01/Socratic-Model-Fine-Tune holds code and JSONL; this repo holds weights.

What is uploaded

Trainer checkpoint-* folders stay on the training PC (socratic_qwen3_v9_model/) and are not uploaded.

Base model

• Instruct / non-thinking checkpoint only (no <think> blocks).
• Do not load these adapters on Qwen3-4B-Thinking-2507.

Train (QLoRA)

4-bit NF4 + LoRA (r=8, alpha=16) via python start.py --qwen → train_qwen.py. Data: Susu11/socraticfinetune.

Deploy (Python / GPU)

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3-4B-Instruct-2507"
adapter = "Susu11/new4b"
tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, adapter)
messages = [
    {"role": "system", "content": "You are a Socratic Science Tutor for a Grade 10 student. Never give the final answer directly."},
    {"role": "user", "content": "Why does ice float?"},
]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0, inputs.input_ids.shape[-1]:], skip_special_tokens=True))

Local helper: uv run python infer_qwen.py after adapters exist.

Deploy (llama.cpp / Ollama)

After merge + convert: python start.py --gguf --qwen. Then point llama.cpp or Ollama at gguf/socratic-qwen3-q8_0.gguf on this repo.

Eval

Same ScienceQA 256-item slice as Phi-3: python start.py --eval --qwen. Compare eval/scienceqa_acc and eval/scienceqa_sri in W&B project science_socratic_qwen3-4b_instruct (WANDB_PROJECT_QWEN).