icdu_dpo_qwen3.5_9b_v2

Model Card for icdu_qwen3.5_9b_dpo_merged_model-q4km

<!-- Provide a quick summary of what the model is/does. -->

Q4_K_M GGUF quantization of a two-stage QLoRA fine-tune of Qwen/Qwen3.5-9B: supervised fine-tuning on ICDU-formatted coaching/instruction data (ICDU General Dataset v8), followed by Direct Preference Optimization for tool-selection behaviour, with both LoRA adapters merged before conversion.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the final artifact of the ai-factory QLoRA + DPO pipeline, built on August 12, 2026, exported to GGUF and quantized to Q4_K_M for llama.cpp-compatible runtimes.

Lineage: (1) a QLoRA LoRA adapter was trained on Qwen/Qwen3.5-9B with ICDU v8 instruction data and merged into the base weights (final_merged_model); (2) a second LoRA adapter was trained on that merged checkpoint with DPO on tool-selection preference pairs and merged again (dpo_merged_model, saved as a text-only Qwen3_5ForCausalLM checkpoint in float16 per its surviving config.json); (3) the merged DPO model was converted to GGUF and quantized to Q4_K_M (llama.cpp general.file_type 15, quantization version 2). The export is text-only; the chat template from the base tokenizer is preserved in the GGUF metadata.

• Developed by: Overture System Solutions
• Model type: Causal language model (QLoRA SFT + DPO fine-tune; GGUF Q4_K_M quantization)
• Language(s) (NLP): English (en)
• License: apache-2.0
• Finetuned from model: Qwen/Qwen3.5-9B

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Model Repository: cds1/icdu_dpo_qwen3.5_9b_v2
• Training Pipeline: https://github.com/Farsalis/ai-factory

How to Get Started with the Model

Use the code below to get started with the model.

# llama.cpp
llama-cli -m icdu_qwen3.5_9b_dpo_merged_model-q4km.gguf \
  -p "I keep failing to build a fitness habit. What should I do?" -cnv

# llama-cpp-python
from llama_cpp import Llama

llm = Llama(
    model_path="icdu_qwen3.5_9b_dpo_merged_model-q4km.gguf",
    n_ctx=8192,  # architecture supports up to 262,144
)
out = llm.create_chat_completion(messages=[
    {"role": "user", "content": "Help me plan my first dinner party."},
])
print(out["choices"][0]["message"]["content"])

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

SFT: ICDU General Dataset v8 - ICDU-formatted JSONL (icdu_training_data_v8.jsonl, 13,671 records; icdu_validation_data_v8.jsonl, 1,520 records). Each record carries the ten-field ICDU schema (persona archetype, governing principle, capability layer, user intent, context summary, prompt, ideal response, attributes, rationale outline) over life/work coaching topics derived from the 'Breaking Better' source corpus.

DPO: 900 chosen/rejected preference pairs generated from bb_training_data_v7.jsonl (tool-augmented chat data). Chosen responses contain the correct tool call; rejected responses either drop the tool call or invoke a synthetically chosen incorrect tool.

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Preprocessing [optional]

ICDU records were rendered to chat format - a system message carrying context, user intent, and persona, plus the user prompt and assistant response - with a 50% chance of persona-specific scenario perturbation; loss was computed on assistant completions only. DPO prompts used the [INST] ... [/INST] instruction format.

Training Hyperparameters

• Training regime: bf16 mixed precision with a 4-bit (NF4) quantized base model for both stages (QLoRA).

Stage 1 - Supervised fine-tuning (SFT):

Stage 2 - Direct Preference Optimization (DPO), via TRL's DPOTrainer:

<!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

Wall-clock training time was not retained from the original run.

Technical Specifications

Model Architecture and Objective

From the GGUF header (general.architecture: qwen35) and the merged checkpoint's config.json (Qwen3_5ForCausalLM, model_type: qwen3_5_text): 9.0B parameters, 32 blocks - 24 linear-attention (Gated DeltaNet SSM: conv kernel 4, 16 key / 32 value heads, head dim 128) and 8 full-attention layers interleaved every 4th layer - hidden size 4096, FFN 12288, grouped-query attention (16 query / 4 KV heads, head dim 256, partial rotary factor 0.25), RoPE base 1e7 with interleaved mRoPE sections [11, 11, 10], context length 262,144, vocabulary 248,320. Source checkpoint dtype float16; GGUF v3.

Compute Infrastructure

[More Information Needed]

Hardware

~5.3 GiB for weights plus KV cache; fits fully offloaded on GPUs with 8 GB VRAM at moderate context, or runs on CPU.

Software

Python 3.12, PyTorch 2.5.1 (CUDA 12.4), transformers 5.12.1 (recorded in the merged checkpoint's config.json), TRL >=0.27 (DPOTrainer), PEFT >=0.18, bitsandbytes >=0.49, datasets >=4.5, as pinned by this repository's environment.yml at build time; GGUF conversion/quantization via llama.cpp (exact version not recorded).

Model Card Authors

Overture System Solutions (O.S.S.) - Jordan Martens, Samuel Conrad.

Model Card Contact

Jordan Martens [email protected]