Downloads · 30 days
228
2% of all-time downloads
TizzyT566/Ornith-1.5-35B-A3B-NVFP4-512K-GGUF
Ornith-1.5-35B-A3B-NVFP4-512K-GGUF is a text generation model from TizzyT566. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This repository contains a GGUF conversion of Ornith AI's Ornith-1.5-35B-A3B-NVFP4 checkpoint for use with compatible GGML/llama.cpp-based inference engines.
Downloads · 30 days
228
2% of all-time downloads
All-time downloads
14.3K
Public
Repo size
24.8 GB
Likes
3
Public
Click a slice to open those files.
.gguf24.8 GB · 100%
From the Hugging Face model README
This repository contains a GGUF conversion of Ornith AI's Ornith-1.5-35B-A3B-NVFP4 checkpoint for use with compatible GGML/llama.cpp-based inference engines.
The purpose of this conversion is to preserve the model's native NVFP4 mixed-precision representation in GGUF rather than requantizing the model into another quantization format.
The model's context length has been extended to 524288 tokens (512K) using YaRN with a ROPE scaling factor of 2, allowing it to handle significantly longer sequences than the original checkpoint.
Base model: ornith-ai/Ornith-1.5-35B-A3B-NVFP4
Architecture: Qwen3.5 MoE
Parameters: 35B total / ~3B active
Quantization: NVFP4
Format: GGUF
Context length: 524288 tokens (512K)
Intended runtime: llama.cpp and other runtimes supporting GGUF NVFP4 tensors
The original Hugging Face Safetensors checkpoint was converted using the convert_hf_to_gguf.py tooling from llama.cpp.
The conversion preserves the source checkpoint's NVFP4 tensors rather than converting the model to FP16, BF16, or FP32 and subsequently requantizing it.
Context length extension to 524288 tokens was applied using YaRN with a ROPE scaling factor of 2.
Ornith-1.5-35B-A3B-NVFP4
│
│ Hugging Face Safetensors
▼
convert_hf_to_gguf.py
│
│ Native GGUF NVFP4 representation
▼
Ornith-1.5-35B-A3B-NVFP4.gguf
│
│ YaRN ROPE scaling factor 2
▼
Ornith-1.5-35B-A3B-NVFP4-512K.gguf
│
▼
llama.cpp
The following settings have been tested on an RTX 5090 running llama.cpp:
ctx-size: 524288
flash-attn: on
temp: 0.6
top-p: 0.95
top-k: 20
image-min-tokens: 1024
reasoning-preserve: enabled
cache-type-k: q8_0
cache-type-v: q8_0