Downloads · 30 days
26
3% of all-time downloads
Ornith-1.0-35B-abliterated-NVFP4
maci0/Ornith-1.0-35B-abliterated-NVFP4
Ornith-1.0-35B-abliterated-NVFP4 is a image-text-to-text model from maci0. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
<div style="font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;max-width:760px;border:1px solid cec9ba;background:f2efe6;color:14130f;padding:24px 26px;border-radius:2px;margin-bottom:26px;"<div style="displ…
All-time downloads
974
Public
Parameters
34.7B
21.9 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors21.9 GB · 100%
Parameter types
How the weights are stored.
U833.6B · 97%
Try a prompt
Base models
Spaces
Model card
From the Hugging Face model README
<div style="font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;max-width:760px;border:1px solid #cec9ba;background:#f2efe6;color:#14130f;padding:24px 26px;border-radius:2px;margin-bottom:26px;"><div style="display:flex;justify-content:space-between;align-items:center;gap:10px 14px;flex-wrap:wrap;"><span style="display:inline-flex;align-items:center;gap:9px;"><svg width="18" height="18" viewBox="0 0 24 24" aria-hidden="true" style="flex-shrink:0;"><path d="M4 4 H16 L20 8 V20 H4 Z" fill="none" stroke="#6f6b60" stroke-width="1.25"/><path d="M16 4 V8 H20" fill="none" stroke="#6f6b60" stroke-width="1.25"/><circle cx="9" cy="13" r="1" fill="#6f6b60"/><circle cx="13" cy="15" r="1" fill="#6f6b60"/></svg><span style="font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:12px;font-weight:700;letter-spacing:0.02em;">RQ-35B-ORN-X</span></span><span style="font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:10px;font-weight:700;letter-spacing:0.12em;text-transform:uppercase;color:#b5231c;border:1.5px solid #b5231c;border-radius:2px;padding:2px 7px;display:inline-block;transform:rotate(-3deg);">Experimental</span></div><div style="font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:26px;font-weight:800;letter-spacing:-0.02em;line-height:1.15;margin:18px 0 8px;color:#14130f;">Ornith-1.0-35B MoE · NVFP4</div><div style="font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;font-size:14.5px;line-height:1.5;color:#4a4740;margin-bottom:18px;">35B MoE (256 experts, 8 active) agentic coder · refusals removed (custom MoE-aware).</div><table style="width:100%;border-collapse:collapse;border:0;margin:2px 0 0;font-variant-numeric:tabular-nums;"><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-top:1.5px solid #14130f;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Params</th><td style="padding:9px 0;border:0;border-top:1.5px solid #14130f;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#14130f;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">35B</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Active</th><td style="padding:9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#b5231c;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">~3B · 8/256 experts</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Size</th><td style="padding:9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#14130f;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">20 GB</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Perplexity</th><td style="padding:9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#14130f;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">7.33</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Refusals</th><td style="padding:9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#b5231c;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">99 → 0 / 8</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">Context</th><td style="padding:9px 0;border:0;border-bottom:1px solid #cec9ba;background:none;text-align:right;font-weight:700;color:#14130f;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">256K</td></tr><tr><th scope="row" style="text-align:left;font-weight:600;padding:9px 8px 9px 0;border:0;background:none;color:#4a4740;letter-spacing:0.01em;font-size:13px;font-family:ui-sans-serif,system-ui,-apple-system,sans-serif;">MTP head</th><td style="padding:9px 0;border:0;background:none;text-align:right;font-weight:700;color:#14130f;font-size:14px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;white-space:nowrap;">n/a</td></tr></table></div>
TL;DR: Ornith-1.0-35B, a 256-expert MoE, quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. 20 GB, wikitext-2 PPL 7.33, agentic coder, refusals removed with a custom MoE-aware method (2 rounds).
[!WARNING] Experimental. This build is mostly a research artifact. The abliteration uses a custom fused-expert MoE method (not a standard tool), validated only on a small informal probe rather than a rigorous benchmark, and NVFP4 quantization of MoE models is bleeding-edge. Expect rough edges. Treat it as a proof of concept, not a production model, and evaluate it yourself before relying on it.
Ornith-1.0-35B abliterated NVFP4
deepreinforce-ai/Ornith-1.0-35B,
a 35B Mixture-of-Experts model abliterated with a custom MoE-aware method (refusal
directions removed from the fused expert tensors), then quantized to NVFP4 (W4A4) in the
compressed-tensors nvfp4-pack-quantized format with
llm-compressor (GPTQ + MSE, shared
fused-layer scales).
Decensored, and compact. Standard abliteration tools cannot reach a fused-expert MoE, so this model was abliterated with a custom per-expert orthogonalization (see Abliteration). NVFP4 then compresses the model to ~20 GB with a wikitext-2 perplexity of 7.33.
| Refusals (held-out probe) | 0/8 harmful prompts it previously refused |
| Abliteration | custom MoE-aware per-expert orthogonalization, 2 rounds |
| Size on disk | ~20 GB vs ~70 GB bf16 (~29%) |
| wikitext-2 PPL | 7.33 |
- Built for vLLM on NVIDIA Blackwell (4-bit weight + 4-bit activation). Pre-Blackwell GPUs run it weight-only.
- Loading and generation verified in vLLM on an NVIDIA GB10 (Blackwell, sm_121).
Uncensored / abliterated model. It follows instructions without refusal guardrails. The abliteration only removes refusals; all other behaviour comes from the base model. You are responsible for how you use it.
Fidelity
Near-lossless versus the bf16 source, 20 GB vs 70 GB bf16 (~29%), at wikitext-2 perplexity 7.33. GPTQ error compensation and an MSE observer keep the drop from bf16 minimal; the header lists the full characteristics and Quantization covers the recipe.
Quickstart
NVFP4 is auto-detected from config.json (compressed-tensors); no quantization flag
needed.
vllm serve maci0/Ornith-1.0-35B-abliterated-NVFP4 \
--served-model-name ornith-35b-abliterated-nvfp4 \
--max-model-len 131072 \
--gpu-memory-utilization 0.90 \
--kv-cache-dtype fp8 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice --tool-call-parser qwen3_coder
- Supports up to 262144 tokens; keep at least 128K to preserve thinking quality.
- Add
--language-model-onlyto skip the vision tower and free KV cache for text use. - The parser flags are not auto-detected; pass them explicitly.
Python (OpenAI client)
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
r = client.chat.completions.create(
model="ornith-35b-abliterated-nvfp4",
messages=[{"role": "user", "content": "Add a retry with exponential backoff to this HTTP client and explain the change."}],
)
print(r.choices[0].message.content)
curl
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "ornith-35b-abliterated-nvfp4",
"messages": [{"role": "user", "content": "Add a retry with exponential backoff to this HTTP client and explain the change."}]
}'
About the base model
Ornith-1.0 is a self-improving family of open agentic-coding models from Deep Reinforce. The 35B member is a Mixture-of-Experts Qwen3.5-MoE-family vision-language model with thinking-mode reasoning and a 256K context.
- 40 decoder layers, Mixture-of-Experts with 256 experts per layer (packed as fused expert tensors), plus a vision tower for image and video input.
- 256K context (
max_position_embeddings262144). - Thinking mode by default, with an instruct toggle (preserved here; abliteration and quantization keep the original chat template).
Abliteration
This is the notable part of this build. Ornith-1.0-35B is a fused-expert MoE: all 256
experts of a layer are packed into a single fused tensor (Qwen3_5MoeExperts.down_proj)
rather than 256 iterable Linear modules. Standard abliteration tooling (including
Heretic) walks the module tree looking for Linear
layers, so it silently skips every expert and can only touch the attention projections.
On this model that leaves a hard refusal floor (roughly 88/100 harmful prompts still
refused), because the refusal behaviour lives in the experts.
To reach the experts, this model was abliterated with a custom MoE-aware method: for each
layer, compute the refusal direction, then orthogonalize the fused experts.down_proj
output per expert against that direction. That is 10240 expert slices (256 experts x 40
layers), applied in two rounds (a second pass removes the residual refusal signal that
remains after the first). Refusal directions are estimated from
mlabonne/harmless_alpaca (good) vs mlabonne/harmful_behaviors (bad).
| Stage | Result |
|---|---|
| Standard tools (Heretic) | attention only, experts skipped, refusal floor ~88/100 |
| MoE-aware round 1 | per-expert orthogonalization of the fused experts.down_proj |
| MoE-aware round 2 | residual refusal direction removed |
| Held-out probe | 0/8 harmful prompts it previously refused |
Honest validation caveat. The result was checked on a small held-out probe: the model no longer refuses 8 harmful prompts it previously refused (0/8). This is a sanity check, not a rigorous refusals-at-fixed-KL benchmark like the smaller dense Ornith build. Treat the decensoring as demonstrated on a small probe, not exhaustively measured.
Quantization
| Scheme | NVFP4, W4A4 |
| Weight rounding | GPTQ (Hessian-based error compensation), MSE observer |
| Weights | FP4 (E2M1), group_size=16, tensor_group, FP8 (E4M3) group scales, shared across fused layers |
| Activations | FP4, dynamic per-group, FP8 (E4M3) scales |
| Quantized | all language-model Linear layers, including the fused MoE expert tensors |
| MoE calibration | moe_calibrate_all_experts=True (every one of the 256 experts must be routed during calibration, or unrouted experts produce garbage) |
| Kept in bf16 | routers (mlp.gate, shared_expert_gate), vision tower (model.visual.*), lm_head |
| Untouched | gated delta-net Conv1d and SSM params (A_log, dt_bias), never Linear |
GPTQ is a quantization-time cost only; inference speed and format are identical to plain round-to-nearest NVFP4, but it chooses better 4-bit values.
Calibration: 512 domain-matched samples (long reasoning + general chat + code),
max_seq_len=2048, text-only path through the VL model, with
moe_calibrate_all_experts=True so all 256 experts per layer receive calibration traffic.
Toolchain: llmcompressor==0.12.0, compressed-tensors==0.17.1, transformers==5.12.1,
torch==2.11.0+cu130, on an NVIDIA GB10 (Blackwell, sm_121). The routers (mlp.gate,
shared_expert_gate) are left in bf16 so routing stays exact.
Recommended sampling
Thinking mode is the default.
- Thinking, precise coding:
temperature=0.6,top_p=0.95,top_k=20 - Thinking, general:
temperature=1.0,top_p=0.95,top_k=20 - Instruct / non-thinking:
temperature=0.7,top_p=0.80,top_k=20 - To run non-thinking, set
{%- set enable_thinking = false %}in the chat template, or passextra_body={"chat_template_kwargs": {"enable_thinking": false}}.
Related
- Base model: deepreinforce-ai/Ornith-1.0-35B
- Space: Rogue Quants
- Collection: NVFP4 Quants
- Sibling NVFP4 quants:
Notes
- Needs NVIDIA Blackwell (sm_121, e.g. GB10) for accelerated W4A4; pre-Blackwell GPUs run it weight-only.
--reasoning-parserand--tool-call-parserare not auto-detected; pass them explicitly.- Thinking mode is on by default; toggle it via the chat template or
chat_template_kwargs. - No refusal guardrails; you are responsible for how you use it.
License
Apache-2.0, following the base model. Intended use and all responsibility for use follow the base model.
Credits
- Base model: Deep Reinforce (Ornith-1.0)
- Abliteration: custom MoE-aware per-expert orthogonalization (fused
experts.down_proj), because standardLinear-walking tools cannot reach fused MoE experts - Quantization tooling: llm-compressor / compressed-tensors
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- qwen3_5_moe
- License
- apache-2.0
- Languages
- en, zh
- Created
- Jul 6, 2026
- Updated
- Aug 4, 2026