Downloads · 30 days
20
26% of all-time downloads
ravensh1212/phonics-story-slm-qwen3-4b-661
phonics-story-slm-qwen3-4b-661 is a text generation model from ravensh1212. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
A PEFT LoRA adapter on Qwen/Qwen3-4B-Instruct-2507 that turns the base model into a phonics-aware early-reading story generator. Given a feasible request, it produces exactly one original six-sentence story whose ever…
Downloads · 30 days
20
26% of all-time downloads
All-time downloads
78
Public
Repo size
144 MB
Likes
0
Public
Click a slice to open those files.
.safetensors132 MB · 92%
From the Hugging Face model README
A PEFT LoRA adapter on Qwen/Qwen3-4B-Instruct-2507 that turns the base model
into a phonics-aware early-reading story generator. Given a feasible request,
it produces exactly one original six-sentence story whose every word is either
a reviewed short-vowel / decodable word or an explicitly approved irregular word,
and whose plot has a complete narrative arc (character, goal, problem, relevant
attempt, causal consequence, satisfying resolution).
This is a controlled-research artifact, not a classroom product. See Intended use & limitations before using it.
Qwen/Qwen3-4B-Instruct-2507, pinned revision
cdbee75f17c01a7cc42f958dc650907174af0554r=16, alpha=32,
dropout=0.05License note:
apache-2.0is chosen to match the base model's permissive license and is a copyright license only — it does not authorize classroom/student or production use (see limitations). Override thelicensefield if your jurisdiction or dataset terms require something different.
The adapter specializes the base model for a single, tightly-scoped behavior
defined by a frozen behavior spec
(config/v3/r4/behavior_spec.json,
spec_id: phonics-story-slm-v3-r4-behavior-20260711). A response must satisfy:
. ! ?.The adapter alone is a strong generator of such stories; guaranteed validity comes from pairing it with the project's deterministic validator (see Deterministic validator & retry).
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj).2e-4, cosine schedule, warmup
ratio 0.03, weight decay 0.0, max grad norm 1.0, effective batch size 16
(micro-batch 8 × grad-accum 2), seed 20260712.r=16, alpha=32, dropout=0.05, bias="none",
task_type="CAUSAL_LM".-100, only the assistant story supervised, a single terminal
<|im_end|> (id 151645) as the final supervised label with no trailing
newline, every serialized row <= 512 tokens (observed max 360).data/v3/r6/train_combined_661/, train SHA-256
00f00bba45ff7de53ce71ca10b63db28b3116daaa7fc6884d0d5b2ea496e078f) — the
pinned 288-row train-only handoff used verbatim as the exact byte prefix
followed by 373 audited r6 mass-expansion rows. Band distribution:
{1:101, 2:130, 3:52, 4:72, 5:97, 6:110, 7:99}. There are no internal
validation/test rows in this file.Trainer + PEFT + bitsandbytes.Reproduce with notebooks/train_qwen3_4b_lora_661_colab.ipynb in the GitHub repo.
The adapter is applied on top of the pinned base revision. Match the base revision exactly for reproducible behavior.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
BASE_MODEL_ID = "Qwen/Qwen3-4B-Instruct-2507"
BASE_MODEL_REVISION = "cdbee75f17c01a7cc42f958dc650907174af0554"
ADAPTER_REPO = "ravensh12/phonics-story-slm-qwen3-4b-661"
tokenizer = AutoTokenizer.from_pretrained(
BASE_MODEL_ID, revision=BASE_MODEL_REVISION, trust_remote_code=False
)
quant_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=torch.bfloat16,
)
base = AutoModelForCausalLM.from_pretrained(
BASE_MODEL_ID,
revision=BASE_MODEL_REVISION,
trust_remote_code=False,
quantization_config=quant_config,
torch_dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(base, ADAPTER_REPO)
model.eval()
messages = [
{"role": "system", "content": "You write one short decodable phonics story."},
{"role": "user", "content": "Write a six-sentence story about a lid on a bin."},
]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)
with torch.inference_mode():
generated = model.generate(
**inputs,
do_sample=False,
max_new_tokens=384,
eos_token_id=151645, # <|im_end|>
pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(generated[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())
The project ships a story-only chat template
(config/story_only_chat_template_v2.jinja) and a curated per-request "binding"
(target word, sense/role, authorized phonics skill+band, scaffold-word
allowlist, goal contract). For the tightest reproduction of the trained behavior,
use those bindings and template from the GitHub repo rather than free-form
prompts.
The model is designed to be run behind a deterministic validator, not
trusted blindly. The project's DeterministicComparisonScorer scores each
candidate story on six dimensions — phonics_legality, target_sense_role,
six_sentence, one_paragraph, narrative, semantic_goal — and a
best-of-K / generate → validate → retry → fail-closed loop releases the first
candidate that passes all dimensions (or refuses). This is how the system
turns a strong-but-imperfect generator into a guaranteed-valid one; see
scripts/generate_with_validator_retry_v1.py and
scripts/run_model_comparison_v3_r4.py in the repo.
Evaluation uses the deterministic phonics validator + narrative/semantic checks (no LLM judge for the pass), comparing the tuned adapter against the exact base checkpoint on a frozen 100-prompt public non-blind set that is disjoint from training (0 exact prompt/story/family overlap; target roots overlap by design).
Numbers below are ILLUSTRATIVE — from the earlier 288-row adapter. They are shown only to indicate the shape of the base→tuned lift. The 661-adapter figures are still being generated and will replace these once the three-way comparison (
scripts/run_model_comparison_v3_r4.py) completes.
| Metric (raw single-shot, greedy) | Base Qwen3-4B | Tuned adapter (288, illustrative) |
|---|---|---|
| Legal-token rate | 71% | 99% |
| Narrative complete | 4% | 68% |
| One-paragraph / format | 40% | 100% |
| Semantic-goal satisfied | 17% | 63% |
The consistent pattern: fine-tuning dramatically raises phonics legality and
format/narrative compliance, with semantic_goal remaining the hardest and
lowest-scoring dimension.
Intended use: controlled research and demos of phonics-constrained, narratively-complete story generation, ideally behind the deterministic validator/retry loop.
Not approved for: classroom, student-facing, or production use, or any learning-outcome / efficacy claim.
Key limitations (see docs/LIMITATIONS_V3.md in the repo for the full list):
semantic_goal check enforces narrow
plan/goal consistency, not real-world common sense or literary quality.data/v3/r6/train_combined_661/ in the GitHub repo
(train SHA-256 00f00bba45ff7de53ce71ca10b63db28b3116daaa7fc6884d0d5b2ea496e078f)config/v3/r4/behavior_spec.json