Downloads · 30 days
77
8% of all-time downloads
l4zy0n3/Ornith-1.5-35B-A3B-ZeroFuse-AD-IQ3S-IQ3XXS-MTP-GGUF
Ornith-1.5-35B-A3B-ZeroFuse-AD-IQ3S-IQ3XXS-MTP-GGUF is a image-text-to-text model from l4zy0n3. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
An importance-matrix quant of junafinity/Ornith-1.5-35B-A3B-uncensored-GGUF-8bit, an abliterated build of ornith-ai/Ornith-1.5-35B-A3B.
Downloads · 30 days
77
8% of all-time downloads
All-time downloads
939
Public
Repo size
17.5 GB
Likes
0
Public
Click a slice to open those files.
.gguf17.5 GB · 100%
From the Hugging Face model README
An importance-matrix quant of
junafinity/Ornith-1.5-35B-A3B-uncensored-GGUF-8bit,
an abliterated build of
ornith-ai/Ornith-1.5-35B-A3B.
This repository is the embedded-MTP edition. The compact imatrix language
model is unchanged; the source model's single Q8_0 MTP block is appended as
blk.40.* and explicitly tested on CUDA.
| File | Purpose | Size |
|---|---|---|
Ornith-1.5-35B-A3B-ZeroFuse-AD-IQ3_S-IQ3_XXS-MTP-Q8.gguf | Language model with embedded Q8_0 MTP | 16.41 GB / 15.27 GiB |
mmproj-Ornith-1.5-35B-A3B-uncensored-f16.gguf | F16 vision projector | 899 MB |
imatrix/imatrix.gguf | Importance matrix used for the compact base quant | 192 MB |
logs/quantize-AD-IQ3_S-IQ3_XXS.log | Quantization log | 123 KB |
tools/merge_mtp_gguf.py | Minimal reproducible MTP merge tool | 3 KB |
benchmarks/100k-recall.json | Exact 100K recall result | 544 B |
SHA256SUMS | Artifact checksums | — |
Download the model and mmproj for image input. The model alone supports text.
| Property | Value |
|---|---|
| Parameters | 35.51B |
| Decoder layers | 40 |
| MTP layers | 1 |
| GGUF tensors | 753 |
F32 tensors | 310 |
Q8_0 tensors | 323 |
IQ3_XXS tensors | 80 |
IQ3_S tensors | 40 |
The MTP block contains 20 blk.40.* tensors copied byte-for-byte from the
Junafinity Q8_0 source. Metadata is updated to block_count=41 and
nextn_predict_layers=1.
Tested with the Muse llama.cpp branch. Builds without embedded draft-mtp
support may load the model normally but will not provide speculative decoding.
llama-server \
--model Ornith-1.5-35B-A3B-ZeroFuse-AD-IQ3_S-IQ3_XXS-MTP-Q8.gguf \
--mmproj mmproj-Ornith-1.5-35B-A3B-uncensored-f16.gguf \
--device CUDA0 --split-mode none \
--n-gpu-layers all \
--n-cpu-moe 16 \
--override-tensor 'blk\.40\..*=CUDA0' \
--ctx-size 102400 --parallel 1 \
--batch-size 256 --ubatch-size 64 \
--threads 8 --threads-batch 8 \
--flash-attn on \
--cache-type-k q4_0 --cache-type-v q4_0 \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
--spec-draft-device CUDA0 \
--spec-draft-ngl all \
--spec-draft-type-k q4_0 \
--spec-draft-type-v q4_0 \
--jinja --reasoning auto --reasoning-format deepseek \
--ctx-checkpoints 1 --checkpoint-min-step 8192 \
--cache-ram 0 \
--host 127.0.0.1 --port 8082
--n-cpu-moe 16 is the tested 16 GB VRAM fit profile, not a universal
recommendation. The MTP layer, target KV, draft KV, and MTP compute buffers were
verified on CUDA0.
Hardware: RTX 4090 Laptop GPU, 16 GB VRAM; i9-14900HX; Linux.
| Gate | Result |
|---|---|
| 8K bounded generation | pass |
| 8K generation | 31.79 tok/s |
| 8K MTP acceptance | 51.1% |
| 100K exact recall | pass |
| 100K prompt processing | 149.74 tok/s |
| 100K generation | 32.57 tok/s |
| 100K MTP acceptance | 60.7% |
| Maximum observed VRAM at 100K | 11,945 MiB |
Exact 100K expected and returned value:
ORCHID-739184, COBALT-482017, SABLE-301955
These are single-machine deployment measurements, not general model-quality benchmarks. Prompt processing with embedded MTP was substantially slower than the best non-MTP deployment on this machine.
The included merge script:
blk.40.* tensors from the Junafinity Q8_0 source.The included importance matrix contains 510 entries computed from 9,686 calibration chunks. The quantization log records the original tensor-level recipe and tool revision.
Ornith-1.5-35B-A3B.No claim is made that quantization or MTP packaging improves source-model intelligence. They trade precision and implementation complexity for local deployment fit and speculative-decoding capability.
The immediate source repository declares Apache-2.0. The original Ornith model declares MIT. Follow all upstream license notices and terms. This model has had safety guardrails reduced or removed. Users remain responsible for legal, safe, and appropriate deployment.