refract-hsmall-blend2

refract-hsmall-blend2 — divergent decision refraction (shipped model)

LoRA adapters that teach ibm-granite/granite-4.0-h-small (32B total / ~9B active, hybrid Mamba+MoE) to refract a hard decision into four genuinely distinct, viable strategic threads — each a different kind of move (confront, evade, co-opt, transform, delegate, endure) — and refuse to blend them into one hedged answer.

This is the shipped result of a concluded 8-round research program. The full record — every round, every number, every mistake, and the negative results — is in the project's working paper: github.com/Nikhiljangra07/divergence-formula.

What's in this repo

Both adapters: LoRA r=64 / α=64, all-linear targets, SFT on 6,745 judge-gated synthetic rows (1,349 decomposer / 5,396 worker), including a targeted "viable-cunning" influence round.

Headline result

Single-session blind judging (Gemini 2.5 Pro, temp 0), 48 out-of-distribution modern decision problems, Claude Haiku 4.5 generated fresh in the same session as the frontier competitor:

It beats Haiku 4.5 on the objective it was built for — distinctness (4.90 vs 4.75) — and ties it on decisiveness, with ~9B active parameters. It loses overall (−0.17); the residue is viability and foresight, and the working paper says so plainly.

Note on the two eval harnesses: the table above is the same-session comparative re-judge (working paper §14–15). The JSON files in eval/ are from the project's internal DAV harness (a stricter per-dimension rubric on the same problem sets) — different scale, same model. Both are documented in the working paper; nothing here is cherry-picked across harnesses.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "ibm-granite/granite-4.0-h-small"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")

# Stage 1: decompose the problem into 4 distinct angles
dec = PeftModel.from_pretrained(model, "Nikhil0097/refract-hsmall-blend2", subfolder="dec_blend2")
# ... generate angles, then per-angle:
# Stage 2: swap to the worker adapter and expand each angle into a full thread

The exact two-stage prompts and generation settings are in the GitHub repo (round2_kit/dav_eval_v5.py). The base model needs the Mamba kernels; the training/inference engineering traps are documented in the working paper §13.

Honest scope

• Specialist scoreboard, not a general benchmark win: the rubric measures the one operation the model was trained for.
• Known weaknesses: viability (3.56 vs Haiku's 4.25) and foresight (4.21 vs 4.52).
• Levers identified but not run (program concluded): hard-negative DPO at 9B-active scale, inference-time viability checking, a foresight-targeted corpus.
• Training data is fully synthetic (DeepSeek V4 generator + LLM judge gate); no human-authored or user data.

Provenance & findings

The program's five load-bearing findings (teachability, quality-beats-size, 3.4B capacity entanglement, entanglement breaking at ~9B active, cheap-generator saturation) are in the working paper. Total program cost: ~$175. Built solo by Nikhil Jangra.