Socrates-Qwen3-8B

Socrates-Qwen3-8B

A QLoRA fine-tune of Qwen3-8B that installs a Socratic reasoning method into the model's <think> trace. Before answering, the tuned model interrogates the question: clarifies what is asked, defines its key terms, surfaces assumptions, tests with counterexamples, and concludes with calibrated confidence, or with explicit aporia when the question cannot be settled.

This repo contains the LoRA adapter (not merged weights). For ready-to-run GGUF quantizations, see Socrates-Qwen3-8B-GGUF. The training data is published openly as Socratic-Reasoning.

What it does

The fine-tune changes the model's reasoning procedure, not its knowledge or style. Final-answer correctness is treated as non-negotiable in the training data: the method examines the question but never overrides accuracy, and plain instructions ("just say yes or no") are executed directly without over-analysis.

Example, "Is 17 prime?"

think: What is 'prime'? Divisible only by one and itself. Does any smaller number divide 17? Not 2 (odd), nor 3, 5, 7; and 7² already exceeds it, so none can remain.

answer: Yes, 17 is prime. Note the answer depended on first fixing what 'prime' means.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "unsloth/Qwen3-8B", device_map="auto", load_in_4bit=True
)
model = PeftModel.from_pretrained(base, "AthrvShrn/Socrates-Qwen3-8B")
tokenizer = AutoTokenizer.from_pretrained("unsloth/Qwen3-8B")

messages = [{"role": "user", "content": "Is a hot dog a sandwich?"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
print(tokenizer.decode(model.generate(inputs, max_new_tokens=1024)[0]))

Or with Unsloth:

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    "AthrvShrn/Socrates-Qwen3-8B", load_in_4bit=True
)

Training

Evaluation

A held-out split (27 examples, zero prompt overlap with train) and a base-vs-tuned scoring harness are defined against these bars:

Scored numbers are not published yet. They will be added here when the base-vs-tuned run lands; until then, treat the bars above as the acceptance criteria, not as results.

Limitations

• Applies the Socratic method broadly by design. It reframes even simple questions. If you want a direct answer, say so; instruction-following is part of the training mix.
• Inherits all limitations and biases of the base model; the fine-tune adds no new knowledge.
• English-focused training data.

License

Apache 2.0, matching the base model.