Downloads · 30 days
1.8K
35% of all-time downloads
georgeis55/Ornith-1.5-35B-A3B-refusal-ablated-MLX-oQ8-mtp
Ornith-1.5-35B-A3B-refusal-ablated-MLX-oQ8-mtp is a machine learning model from georgeis55. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as mit.
This model was quantized using oQ (oMLX v0.6.3rc3) mixed-precision quantization.
Downloads · 30 days
1.8K
35% of all-time downloads
All-time downloads
5.2K
Public
Parameters
36B
75.9 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors38.6 GB · 100%
How the weights are stored.
U3235.5B · 99%
From the Hugging Face model README
This model was quantized using oQ (oMLX v0.6.3rc3) mixed-precision quantization.
Refusal-ablated Ornith-1.5-35B-A3B with the official Ornith MTP head grafted in at 8-bit, so Lightning-MTP speculative decoding works. The refusal-ablated derivative ships without an MTP head; this build restores it from the upstream ornith-ai/Ornith-1.5-35B-A3B checkpoint and quantizes it in the same oQ8 pass as the body.
temperature = 0.6
top_p = 0.95
top_k = 20
| Context | Prefill (tok/s) | Decode (tok/s) |
|---|---|---|
| 1K | 1,670 | 81.8 |
| 4K | 2,084 | 76.1 |
| 8K | 1,965 | 85.1 |
| 16K | 1,738 | 90.2 |
| 32K | 1,402 | 79.2 |
Base weights were modified by directional ablation to suppress refusal behaviour. This model will produce harmful, illegal, or dangerously incorrect content more readily than the upstream instruction model. You own the moderation layer the original guardrails carried, and remain responsible for the base model's licence and applicable law.