qwen3-decodable-story-sft

Qwen3 Decodable Story SFT

Overview

Qwen3 Decodable Story SFT is a QLoRA fine-tuned version of Qwen3-0.6B developed to generate decodable reading stories for beginning readers.

Given one or more target phonics patterns, the model generates a short story intended to emphasize those patterns while maintaining a coherent beginning, middle, and ending.

This project explores whether supervised fine-tuning on a relatively small, curated dataset can teach a reliable educational behavior to a small language model.

Model Details

• Base model: unsloth/Qwen3-0.6B-unsloth-bnb-4bit
• Fine-tuning method: QLoRA (Unsloth)
• Training framework: TRL SFTTrainer
• Task: Decodable story generation

Supported Phonics Patterns

• Short-vowel words
• Consonant blends
• Consonant digraphs
• Final-e words
• Vowel teams
• R-controlled vowels
• Diphthongs
• Multisyllabic words

Evaluation

The model was evaluated against the base model using held-out prompts.

Evaluation consisted of:

• Objective phonics-pattern compliance metrics
• Blind LLM-as-a-judge scoring
• Qualitative error analysis

The fine-tuned model demonstrated improved adherence to the requested phonics behavior compared with the base model while continuing to exhibit limitations on more complex phonics combinations.

Intended Use

This model is intended for:

• Educational NLP experiments
• Decodable story generation
• Small language model fine-tuning demonstrations
• Research on behavior-specific supervised fine-tuning

Limitations

This is a research prototype and is not intended for instructional or production use.

Performance is stronger on simpler phonics patterns than on prompts requiring multiple or more complex phonics constraints. Additional curated training data would likely improve consistency.

Example Prompt

Write a decodable story.

Emphasize these phonics patterns:
- final-e words
- consonant blends

Example words containing these patterns:
- final-e words: cake, game, bike, time, home
- consonant blends: stop, frog, black, plant, grin

Write a coherent narrative with a clear beginning, middle, and ending.

Include a natural story title.

Do not mention phonics terms or pattern names in the title or story.