Ornstein3.6-35B-A3B-SABER-GGUF

Ornstein3.6-35B-A3B-SABER — GGUF

GGUF quantizations of DJLougen/Ornstein3.6-35B-A3B-SABER for use with llama.cpp, ollama, LM Studio, and compatible runtimes.

Source model is the SABER-ablated variant of Ornstein3.6-35B-A3B (Qwen3.5 MoE, 35B total / ~3B active). See the source model card for a description of SABER.

Support This Work

I'm a PhD student in visual neuroscience at the University of Toronto who also happens to spend way too much time fine-tuning, merging, and quantizing open-weight models on rented H100s and a local DGX Spark. All training compute is self-funded — balancing GPU costs against a student budget. If my uploads have been useful to you, consider buying a PhD student a coffee. It goes a long way toward keeping these experiments running.

Support on Ko-fi

---

Quantization suite (8-bit and under)

All variants derived from the bf16 SABER safetensors via llama.cpp convert_hf_to_gguf.py → llama-quantize. Non-Q8_0 K-quants are derived from the Q8_0 file with --allow-requantize.

Active parameters per token are ~3B regardless of file size; the table reflects total weights on disk.

Usage (llama.cpp)

./llama-cli -m Ornstein3.6-35B-A3B-SABER-Q4_K_M.gguf \
    -p "You are a helpful assistant." \
    -cnv --temp 0.7 --top-p 0.9

Intended use

Research and red-teaming. The SABER-ablated model complies with requests its parent model refused. Deploy behind your own policy/logging layer.

License

Apache 2.0, inherited from the base model.