Harmonic-9B-GGUF

☕ Support This Work

I'm a PhD student in visual neuroscience at the University of Toronto who also happens to spend way too much time fine-tuning, merging, and quantizing open-weight models on rented H100s and a local DGX Spark. It's a hobby that got out of hand. If my uploads have been useful to you, consider buying a PhD student a coffee. It goes a long way toward keeping these experiments running.

☕ ko-fi.com/djlougen

Harmonic-9B-GGUF

GGUF quantizations of Harmonic-9B for local inference with llama.cpp, Ollama, LM Studio, and other GGUF-compatible runtimes.

Harmonic-9B is a reasoning backbone — not a general chat model, not an agent, and not a casual assistant.

It was built to think with real structure: consistent self-correction, verification steps, and multi-path exploration in every training example.

Stage 1 (this model): Heavy reasoning fine-tune on privately generated, structurally validated data. Every row passes strict quality gates. No junk, no shallow traces.

Stage 2: Harmonic-Hermes-9B — the dedicated agent/tool-calling variant, fine-tuned on curated tool-calling and agent interaction data.

Available Quantizations

Recommended Quant

Q5_K_M for most users - fits in 8GB VRAM with room for context, minimal quality degradation on reasoning tasks.

Q8_0 if you have the VRAM - preserves the full reasoning depth that the model was trained for.

Vision (Multimodal)

This model includes Harmonic-9B-BF16-mmproj.gguf - the vision projector for multimodal inference. Use with llama.cpp's --mmproj flag for image understanding tasks.

Training Approach

1,817 curated rows. The premise is simple: a small dataset with perfect structural quality beats a massive one full of noise. Rather than training on tens of thousands of unfiltered examples, every row was built to reinforce genuine reasoning behavior.

Every training row contains explicit self-correction ("wait, that's not right"), verification ("let me check by plugging back in"), and multi-path exploration ("alternatively, I could try..."). The data was generated from multiple frontier models and filtered through a custom structural quality pipeline that enforces reasoning depth, coherence, and flow patterns. 100% of rows pass all quality gates simultaneously.

A small set of everyday conversation data is mixed in to preserve the base model's conversational ability - calibrated by token ratio analysis to prevent the reasoning data from drowning out conversational patterns during training.

Training Data Quality

The reasoning data was curated using a custom structural process supervision pipeline. Key metrics:

Every row was scored across multiple structural dimensions and only rows passing all thresholds simultaneously were included. No rows were manually curated - the pipeline is fully automated and reproducible.

How It Compares

We ran our structural quality analysis against every major public reasoning dataset used for Opus/Qwen distillation. The results:

The popular Opus distillation datasets (Crownelius, nohurry, TeichAI) have less than 1% quality gate pass rate. Their thinking traces average under 200 words with near-zero self-correction. Models trained on this data learn to produce short, shallow chain-of-thought that looks like reasoning but lacks the structural behaviors that make reasoning reliable.

Jackrong and Stratos are closer competitors but still fall short on consistency. Jackrong has massive traces (6,653 words avg) but only 22.7% pass the quality gate - the thinking is verbose but wanders. Stratos has decent markers but 49% of rows still fail, meaning half the gradient updates during training push the model toward shallow patterns.

Harmonic's data is smaller by design. Every row passes. Every gradient update reinforces genuine reasoning behavior.

Reasoning Flow

Marker density measured across 20 equal segments of each thinking trace. The characteristic curve shows reasoning intensity building through the middle of the trace and peaking in the later segments as the model enters verification and self-correction before committing to an answer.

Training Configuration

base_model: Qwen/Qwen3.5-9B
dataset: 1,459 reasoning + 358 conversation rows
epochs: 1
learning_rate: 1e-4
lr_scheduler: cosine
warmup_ratio: 0.1
max_seq_length: 8192
lora_rank: 32
lora_alpha: 32
dropout: 0.05
micro_batch_size: 1
gradient_accumulation_steps: 4
weight_decay: 0.01

Usage

Ollama

ollama run DJLougen/Harmonic-9B-GGUF

llama.cpp

./llama-cli -m Harmonic-9B-Q5_K_M.gguf -p "Solve this step by step..." -n 4096

LM Studio

Download any quantization and load in LM Studio. The model follows standard ChatML formatting.

Reasoning format

The model uses <think> blocks for reasoning:

<think>
The user is asking about X. Let me consider two approaches...

Approach 1: ...
Approach 2: ...

I'll go with Approach 1 because...

Wait, I need to be careful here - this assumes Y, which may not hold.
Let me verify by checking a special case...

Yes, that confirms the result.
</think>

[Final answer here]

Intended Use

• Reasoning tasks requiring genuine multi-step thinking
• Mathematical problem-solving with self-correction
• Code analysis and generation with structured verification
• General conversation (conversational ability preserved through training design)
• Base model for Stage 2 agentic fine-tuning

Limitations

• 9B parameter model - not suitable for tasks requiring extensive world knowledge
• Reasoning traces can be verbose for simple questions
• Not optimized for tool calling — see Harmonic-Hermes-9B for agentic use
• Benchmark evaluation is ongoing

Architecture

• Base: Qwen 3.5 9B (9.65B parameters)
• Training: LoRA fine-tuning, merged into base weights
• Precision: BF16
• Context: 8192 tokens

License

Apache 2.0 - same as the base model. All training data is from Apache 2.0 or MIT licensed sources. Fully commercial use permitted.

Links

• Full model weights: DJLougen/Harmonic-9B
• Agentic variant: DJLougen/Harmonic-Hermes-9B