Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK

<h1 align="center">Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK</h1> <p align="center"> <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License: MIT"></a> <a href="https://github.com/vllm-project/llm-compressor"><img src="https://img.shields.io/badge/Quantization-FP8__BLOCK-blue.svg" alt="FP8"></a> <a href="https://huggingface.co/protoLabsAI/Ornith-1.0-9B-MTP"><img src="https://img.shields.io/badge/MTP-Speculative_Decoding-purple.svg" alt="MTP"></a> <a href="https://heretic-project.org/"><img src="https://img.shields.io/badge/Heretic-v1.2.0-red.svg" alt="Uncensored"></a> </p> <p align="center"><b>FP8 Block Quantization + MTP Speculative Decoding</b></p> <p align="center">Uncensored | 35B MoE (3B active) | Vision + Text | ~38GB</p> <p align="center"> <a href="https://huggingface.co/llmfan46/Ornith-1.0-35B-uncensored-heretic">Base Model</a> · <a href="https://huggingface.co/shisa-ai/Ornith-1.0-35B-FP8-BLOCK-MTP">FP8 Reference</a> · <a href="https://huggingface.co/protoLabsAI/Ornith-1.0-9B-MTP">MTP Technique</a> </p>

---

Overview

This is an FP8_BLOCK quantized version of llmfan46/Ornith-1.0-35B-uncensored-heretic with an MTP head grafted from Qwen3.6-35B-A3B for speculative decoding.

The uncensored heretic version achieves 90% fewer refusals (9/100 vs 89/100 original) with only 0.0019 KL divergence.

Key Features

<table> <tr> <td><b>Quantization</b></td><td>FP8_BLOCK (128×128 block, data-free PTQ)</td> </tr> <tr> <td><b>Model Size</b></td><td>~38GB (vs ~70GB BF16)</td> </tr> <tr> <td><b>MTP Head</b></td><td>BF16, from Qwen3.6-35B-A3B (19 tensors)</td> </tr> <tr> <td><b>Throughput</b></td><td>+21% with speculative decoding (ref. benchmark, not tested)</td> </tr> <tr> <td><b>Architecture</b></td><td>Qwen3.5 MoE, 256 experts, 8 active per token</td> </tr> <tr> <td><b>Multimodal</b></td><td>Vision + Text</td> </tr> </table>

---

Quick Start

vLLM (Recommended)

pip install vllm

vllm serve AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --served-model-name ornith-35b-uncensored-fp8 \
  --quantization compressed-tensors \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.93 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Performance-optimized configuration (validated on RTX 4090 48GB):

vllm serve AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --served-model-name ornith-35b-uncensored-fp8 \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 204800 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.93 \
  --dtype bfloat16 \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8_e4m3 \
  --mm-encoder-tp-mode data \
  --mm-processor-cache-type shm \
  --mm-shm-cache-max-object-size-mb 512 \
  --media-io-kwargs '{"video": {"num_frames": -1}}' \
  --attention-backend flashinfer \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --generation-config vllm \
  --max-cudagraph-capture-size 16 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --compilation_config.mode VLLM_COMPILE \
  --compilation_config.cudagraph_mode PIECEWISE \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_xml \
  --enable-auto-tool-choice \
  --trust-remote-code

GPU Requirements: ~40GB VRAM minimum (model is ~38GB FP8). The configuration above was validated on a modified RTX 4090 48GB. For lower VRAM GPUs, quantization to GGUF format is recommended instead.

SGLang

pip install sglang

python3 -m sglang.launch_server \
  --model-path AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --host 0.0.0.0 \
  --port 30000

Transformers

from transformers import AutoModelForMultimodalLM, AutoProcessor

model = AutoModelForMultimodalLM.from_pretrained(
    "AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK",
    trust_remote_code=True,
)

---

Performance

Note: This model has not been benchmarked. The data below is from the reference model shisa-ai/Ornith-1.0-35B-FP8-BLOCK-MTP and is provided for reference only.

Reference benchmarks (RTX PRO 6000 Blackwell, single GPU):

Source: shisa-ai/Ornith-1.0-35B-FP8-BLOCK-MTP

---

Quantization Details

• Tool: llmcompressor v0.12.0
• Method: Model-free PTQ (no calibration data needed)
• Scheme: FP8_BLOCK (compressed-tensors format)
• Weight: Static FP8, symmetric, 128×128 block strategy
• Activation: Dynamic FP8, symmetric, group size 128
• MTP Head: BF16 (not quantized)

Ignored layers: lm_head, embed_tokens, visual.*, mlp.gate, mlp.shared_expert_gate, linear_attn.*, mtp.*

---

Model Lineage

deepreinforce-ai/Ornith-1.0-35B (MIT, original)
    └── llmfan46/Ornith-1.0-35B-uncensored-heretic (Heretic v1.2.0 MPOA ablation)
        └── AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK (this model)
            ├── FP8_BLOCK quantization (llmcompressor)
            └── MTP head grafted from Qwen/Qwen3.6-35B-A3B (donor only)

---

Credits

---

License

MIT — same as the source Ornith-1.0-35B model.

Citation

@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}