Downloads · 30 days
115
9% of all-time downloads
jhone888/Ornith-1.5-35B-A3B-FULL-OBLITERATED-NVFP4
Ornith-1.5-35B-A3B-FULL-OBLITERATED-NVFP4 is a text generation model from jhone888. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
NVFP4 quantization of jhone888/Ornith-1.5-35B-A3B-FULLY-OBLITERATED — the second-pass obliterated v2, now a structurally complete checkpoint: the language model, the native MTP head (self-drafting speculative decoding…
Downloads · 30 days
115
9% of all-time downloads
All-time downloads
1.3K
Public
Parameters
21.6B
44.3 GB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors23.4 GB · 100%
How the weights are stored.
U816.4B · 76%
From the Hugging Face model README
NVFP4 quantization of jhone888/Ornith-1.5-35B-A3B-FULLY-OBLITERATED — the second-pass obliterated v2, now a structurally complete checkpoint: the language model, the native MTP head (self-drafting speculative decoding, 785 tensors) and the vision tower (333 tensors), transplanted byte-identical from the v1 base.
⚠️ Uncensored model. You are responsible for its use.
| Component | Format |
|---|---|
| Routed experts (un-fused, 256/layer) | W4A16_NVFP4, group-16 |
| Shared experts, lm_head | W4A16_NVFP4, group-16 |
| Attention / linear-attention projections | FP8 (per-tensor weight scale) |
| MTP head, vision tower, norms, embeddings, conv | BF16 (untouched) |
| KV cache | FP8 (config flag) |
23.4 GB across 5 shards. Tensor-level audit against the official ornith-ai/Ornith-1.5-35B-A3B-NVFP4 recipe: weight_scale_2 = 30,841 and 291 quantized layers (FP8 130 + W4A16 161) — exact match. FP8 layers use static weight scales (no activation input_scale); verified working with the FreeToken FTW pipeline.
This repo ships the full multimodal config (model_type: qwen3_5_moe, nested text_config/vision_config) because the checkpoint genuinely contains the MTP head and vision tower. vLLM and transformers load it as-is.
FreeToken's ft checkpoint (FTW conversion) needs one change: swap config.json for the flat text-only variant, or the conversion fails with Missing MoE expert source layers. Why: with the multimodal config, FreeToken's model adapter dispatches expert loading to its BF16-bank provider, which searches for fused expert banks (model.layers.N.mlp.experts.gate_up_proj) that don't exist in any NVFP4 build (experts are stored un-fused and quantized per-expert). The flat text-only config (model_type: qwen3_5_moe_text, all text fields at top level) dispatches to the NVFP4 provider, which reads the un-fused quantized layout correctly.
The drop-in flat config is pottokao's config.json with one mandatory edit: architectures must be ["Qwen3_5MoeForConditionalGeneration"] (FreeToken's registry rejects ...ForCausalLM). The MTP/vision tensors are then ignored by FreeToken's text-only FTW; keep this folder's config for vLLM and other engines.