1b-sft-dataset

1B-SFT — High-Quality Instruction Dataset for a ~1B LLM

A curated, deduplicated instruction-tuning dataset in ShareGPT format, built for fine-tuning a 1B-class model (Qwen2.5-1.5B, Llama-3.2-1B, Gemma-2-2B).

5,235 samples — train 4,919 / val 158 / test 158.

Quality guarantees

• Math: every answer is computed by the generator (arithmetic, percentages, word problems, multi-step problems, unit conversions, linear equations, sequences, fractions) with step-by-step working.
• Coding: every code sample is executed and unit-tested before inclusion (33 task types + 10 bug-fix tasks); the shipped code blocks were re-executed against the final files (1,047/1,047 pass).
• Reasoning: ordering puzzles, liar-guard puzzles, syllogisms, seating puzzles — all consistent by construction.
• Hand-written core: ~260 manually authored seeds across knowledge, explain, creative writing, roleplay (flirty companion, boxing coach, motorcycle buddy, DM), advice, summarization, rewriting, and multi-turn chat.
• Exact + SimHash near-duplicate removal; artifact filtering; deterministic split (94/3/3).

Format (ShareGPT, one JSON per line)

{"system": "optional persona...",
 "conversations": [
   {"from": "human", "value": "..."},
   {"from": "gpt", "value": "..."}]}

Category breakdown

Loading

from datasets import load_dataset
ds = load_dataset("andro124543/1b-sft-dataset", split="train")

or use train.jsonl / val.jsonl / test.jsonl directly with LLaMA-Factory (sharegpt formatting) or axolotl.

Suggested recipe for a 1B model (SFT)

• Base: Qwen/Qwen2.5-1.5B (ChatML) or meta-llama/Llama-3.2-1B
• lr 2e-5 cosine, batch 8–16, 3 epochs, max_seq_len 1024–2048, packing on
• QLoRA 4-bit (rank 32) for limited compute; export to GGUF for Ollama

Rebuild / extend

The dataset is fully reproducible: content.py (hand-written seeds), generators.py (verified programmatic generators), build.py (dedup → filter → split → stats). Run python build.py to regenerate deterministically.