Ornstein-9Bv2.5-NSC-ACE

Ornstein 9B v2.5 — NSC-ACE

An NSC-ACE post-train (Neural Steering Committee for Agentic Co-Evolution) of Ornstein 3.5 9B V2.5, targeting agentic tool calling. During GRPO rollout generation, contrastive hidden-state steering directions (agentic vs. passive prompts, recomputed from the live policy) are injected at per-rollout strengths (-0.3, 0.0, 0.3, 0.6) across the last 18 transformer layers, turning each prompt's rollout group into a committee over latent modes. The reward stack is anchored by a per-row declarative validator — expected tool-call names, required arguments, and argument-value constraints checked in-process — with steered-rollout self-consistency as the structure-convergence signal. Like the rest of the line, NSC-ACE keeps the native vision tower and the multi-token-prediction (MTP) head for speculative decoding.

Status: experimental checkpoint. Released with full weights + GGUFs for community testing. See Evaluation for an honest account of what is and isn't yet measured.

Evaluation

GBS STANDARD-200 — avg@3 with sampled decoding (temperature 0.6 / top-p 0.95 / top-k 20), a single identical methodology applied to both models, both served by the same vLLM 0.24 + MTP stack (no published-number carryover). Comparison restricted to the 150 items shared by both runs — HLE, LiveCodeBench, HLCE; GPQA excluded (see caveat):

Honest read: a statistical tie. Item-level paired deltas: overall +0.013 (p=0.52), coding +0.030 (p=0.54), reasoning 0.000 (p=1.0). The LiveCodeBench gain (0.133 → 0.173) is not significant at n=50, and nothing regresses. Decoding methodology now matches the V2.5 card's evaluation (avg@3, identical sampling parameters), so this table can be read in the same register as that card's plot — subject to the composition caveat below.

Composition caveat: HLE and HLCE ran on small fixture sets in this run (no public loader exists for either); fixture columns saturate near 1.0 and do not discriminate. LiveCodeBench is real data (code_generation_lite) and is scored by executing each generated program against the dataset's public stdin/stdout test cases — a scoring change from the earlier assert-based fixture path. GPQA is excluded entirely: the base leg resolved to fixtures (gated-dataset access landed only after it completed), so no like-for-like GPQA pair exists. Only LiveCodeBench provides discriminative real-data signal here.

An earlier single-pass greedy paired run (delta 0.000 across all categories — no regression) is superseded by this avg@3 evaluation.

Training-side signals (right panel above; per-step means over 8 steered rollouts, quartile means Q1 → Q4):

Noisy but real upward trends; the self-consistency gain is the method's structure-convergence thesis showing up in training.

Local behavioral probes (Q8_0 GGUF, llama.cpp b9940, RTX 3090): given Hermes <tools> signatures, the model selects the correct function among distractors, infers unstated argument values ("NVIDIA … in euros" → {"symbol": "NVDA", "currency": "EUR"}), emits exact <tool_call> grammar with think-first reasoning, and declines to call tools on questions where they're irrelevant.

Release line

• V1 — initial reasoning fine-tune.
• V1.5 — refined supervised fine-tune on quality-gated reasoning data.
• V2 — reinforcement-learning post-training (DPO + GRPO verifiable-reward RL).
• V2.5 — strided EBFT on a balanced AgentWorld-oracle + reasoning blend.
• NSC-ACE — this release — Neural Steering Committee + validator-anchored GRPO on V2.5, targeting agentic tool calling.

Quantizations

• GGUF (llama.cpp, MTP draft head embedded): DJLougen/Ornstein-9Bv2.5-NSC-ACE-GGUF — F16, Q8_0, Q6_K, Q5_K_M, Q4_K_M.

Usage

vLLM (recommended — with MTP speculative decoding)

vllm serve DJLougen/Ornstein-9Bv2.5-NSC-ACE \
  --max-model-len 8192 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

Sample with the Qwen3.5 reasoning defaults: temperature=0.6, top_p=0.95, top_k=20.

llama.cpp (MTP speculative decoding, head embedded in the GGUFs)

llama-server -hf DJLougen/Ornstein-9Bv2.5-NSC-ACE-GGUF:Q8_0 \
  -ngl 99 --spec-type draft-mtp

Measured on an RTX 3090 at Q8_0: 67.6 → 100.7 tok/s on code (+49%), 90.6 → 110.9 tok/s on prose (+22%) vs. non-speculative.

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "DJLougen/Ornstein-9Bv2.5-NSC-ACE"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "Search for the latest CUDA release notes and summarize the key changes."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=1024, temperature=0.6, top_p=0.95, top_k=20, do_sample=True)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Tool-call grammar

Trained on Hermes-style calls with per-row tool exposure (<tools> JSON signatures in the system turn):

<tool_call>
{"name": "function_name", "arguments": {"param": "value"}}
</tool_call>

Training recipe

• Method: GRPO (TRL 1.7.1) with a Neural Steering Committee — contrastive hidden-state directions from agentic vs. passive prompts, injected during rollout generation at strengths (-0.3, 0.0, 0.3, 0.6) over the last 18 layers; directions recomputed from the live policy.
• Rewards (weights): validator reward 0.35 (F1 of predicted tool calls against per-row declarative validators: names + required args + eq/in/gte/lte/regex/contains value constraints) · self-consistency 0.20 · process reward 0.15 · format 0.10 · tool-count 0.10 · reasoning-depth 0.05 · diversity 0.05.
• Adapter: QLoRA on NF4 4-bit base — r=64, α=128, dropout 0.05, q/k/v/o/gate/up/down (116M trainable, 1.28%), merged to bf16 for release; original MTP head and vision tower re-attached unchanged (MTP is draft-then-verify, so a frozen draft head is output-exact).
• Schedule: 200 steps, 8 rollouts/prompt, max 768 completion tokens, lr 5e-6, 10% warmup, KL β 0.01 (observed KL ~0.001–0.002); 2 h 37 m on 1× A100 SXM4 80GB.
• Data: NSC-ACE validator dataset — 2,000 rows converted from DJLougen/Acta (Hermes function calling); each row carries its verbatim <tools> system turn, reference <tool_call> blocks, and an auto-generated declarative validator. 200-row held-out eval split.
• Base: DJLougen/Ornstein-3.5-9B-V2.5 (Qwen3.5-9B lineage).

Details

• Developed by: DJLougen / GestaltLabs
• Base model: DJLougen/Ornstein-3.5-9B-V2.5
• Post-training: NSC-ACE (steering committee + validator-anchored GRPO, LoRA merged)
• Parameters: ~9.65B
• Precision: BF16
• Format: ChatML / Qwen3.5 chat template (reasoning <think> enabled)
• Capabilities retained: vision tower (image/video), MTP head (speculative decoding)
• License: Apache 2.0

Support This Work

All training compute is self-funded on rented GPUs and a local DGX Spark. If these uploads are useful, consider buying a PhD student a coffee — it keeps the experiments running.

Support on Ko-fi

License

Apache 2.0 — inherited from the Qwen 3.5 9B base release.