Ornith-1.0-35B-Q4_K_M-MoEspresso

Ornith 1.0 35B Q4_K_M, MoEspresso package

This repository holds a MoEspresso package, and MoEspresso is the only engine that serves it: https://github.com/steadfastgaze/MoEspresso.

The weights are GGUF K-quant tensors repacked into safetensors shards, described tensor by tensor in a content-hashed manifest.

llama.cpp, transformers, and mlx-lm will not load these files.

Install and serving instructions live in the MoEspresso repository.

Original model: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B

Quality

The answer checks used fixed seeds and the model author's recommended sampling settings.

The numerical reference above is the source Q4_K_M GGUF, not the original bf16 model.

Speed

Measured on an Apple M3 Max with 128 GB of unified memory, q8 KV cache - MoEspresso engine.

Longest context reached in testing: 160,965 tokens, grown in one session.

Recipe

The package is built from three inputs.

qwen_kquant_recipe_report.json sidecar records the full recipe, including input hashes and per-shape codec assignments. package_manifest.json records the sha256 of every file; after downloading, moespresso-verify checks the package against it in one command.

The routed experts ship byte for byte from bartowski's Q4_K_M file: gate and up projections in q4_k on all 40 layers, down projections in q4_k on 20 layers and q6_k on the other 20. Nothing in those tensors was re-quantized.

The 252 dense tensors take a different path. They are encoded fresh from the bf16 release under bartowski's imatrix, with a heavier codec mix: 94 tensors in q8_0, 96 in q6_k, 10 in q5_k, 52 in q4_k. Another 361 small tensors (norms, biases) pass through in f32.

Files

Six safetensors shards (19.91 GiB, 733 tensors) plus the sidecars the engine reads: package_manifest.json (per-file sha256 and per-tensor decisions), package_plan.json (the allocation the writer consumed), agentic_profile.json (validated tool-calling defaults), and expert_hotlist.json (cold-start expert prewarm ranking). Tokenizer files and the chat template are vendored from the original release.

Credits

This quantization is derived from bartowski's quantization work; the expert bytes in this package ship unchanged and the imatrix guiding the dense encoding as well. The model is Ornith 1.0 by deepreinforce-ai. Everything in the chain is MIT: the original release, bartowski's GGUFs, and this package.