Huihui-Ornith-1.5-35B-A3B-Uncensored-NInfer

Huihui-Ornith-1.5-35B-A3B Uncensored — ready-to-run NInfer artifact

A single .ninfer file (~22.8 GB) that runs huihui-ai/Huihui-Ornith-1.5-35B-A3B-abliterated (uncensored 35B-A3B MoE) on one RTX 5090 (32 GB) at the model's full 262,144-token context, with 4-bit K8V4 KV cache and MTP speculative decoding, using the NInfer engine.

Full build guide + scripts: pcmaker/Huihui-Ornith-1.5-35B-A3B-Uncensored-NInfer-RTX5090-Guide.

Authorized use only. These are uncensored / abliterated weights: the model's built-in refusals are removed. Use only for red-teaming and security testing of your own systems, safety research, and offline experimentation. You are responsible for what you generate.

Measured on an RTX 5090

• Full 262,144 context loads; ~7,412 tok/s prefill on a 184,910-token prompt.
• Decode ~510 tok/s @185k, 600-850 tok/s on short prompts (MTP, ~76-83% accept).
• Needle-in-haystack at 185k: passed. Model load ~5 s.

Quick start

# 1) build NInfer (Ubuntu 24.04, CUDA 13.3, RTX 5090) — see the guide repo
git clone --depth 1 https://github.com/Neroued/ninfer.git && cd ninfer
export CUDA_HOME=/usr/local/cuda-13.3 PATH="/usr/local/cuda-13.3/bin:$PATH"
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release && cmake --build build -j 6
export LD_LIBRARY_PATH=/usr/local/cuda-13.3/lib64:$LD_LIBRARY_PATH

# 2) download this artifact
hf download pcmaker/Huihui-Ornith-1.5-35B-A3B-Uncensored-NInfer --local-dir ./art

# 3) serve (OpenAI/Anthropic-compatible on :8188)
./build/apps/ninfer-serve ./art/ornith_35b_a3b.ninfer \
  --host 0.0.0.0 --port 8188 --model-id ornith-35b \
  --max-context 262144 --kv-dtype k8v4 --spec mtp --draft-tokens 3 \
  --default-thinking-budget 8192

curl http://localhost:8188/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"ornith-35b","messages":[{"role":"user","content":"Merhaba, kimsin?"}],
       "max_tokens":100,"temperature":0.6,"top_p":0.95,"top_k":20,"reasoning_effort":"none"}'

Notes

• Recommended sampling: temperature=0.6, top_p=0.95, top_k=20 (general use).
• Thinking is on by default; send {"reasoning_effort":"none"} to disable per request.
• Speculative: use --spec mtp (self-aligned, best on this base). --spec dflash --draft-tokens 7 --lm-head-draft is possible but usually lower accept.
• The artifact must be served on the same NInfer version it was built with. If you see tensor descriptor does not match target contract, rebuild from current master and re-convert (see the guide).

Credits

Ornith team, huihui-ai, the Qwen team, z-lab, and the NInfer author (Neroued). All weights and tools belong to their respective authors and licenses.