VoronoiReasoner-41M-IT-GGUF

VoronoiReasoner-41M-IT (Native C++ & Multi-Quant GGUF)

VoronoiReasoner-41M-IT is an ultra-compact 41.46M parameter instruction-tuned reasoning model designed for ultra-high-speed edge inference, local servers, and real-time interactive agents.

By combining continuous hypersphere embeddings with hybrid double-gated short convolutions and grouped-query attention, VoronoiReasoner delivers ultra-low latency (0.75 ms/tok) and blazing throughput (1,200 – 1,328+ tokens/sec) on single-batch ($B=1$) interactive generation with a memory footprint of only ~36 MB VRAM.

---

Key Highlights

• ⚡ Blazing Native C++ Speed: Generates at 1,200 to 1,328+ tokens/second on single-batch ($B=1$) with sub-millisecond response latency (0.75 ms/token) on modern RTX GPUs.
• 💾 Microscopic Memory Footprint: Quantized GGUF models range from 34 MB to 42 MB, consuming under 50 MB VRAM with full GPU offloading.
• 🛠️ Multi-Quantization Support: Native GGUF builds available in Q4_K_M, Q8_0, IQ4_XS, and F16.
• 🎯 Commonsense Reasoning: Achieves 30.00% HellaSwag accuracy (+5.00% over random baseline) in only 41M parameters.
• 💬 Instruction Tuned: Fine-tuned on multi-turn conversational data (HuggingFaceH4/ultrachat_200k) with ChatML format.
• 🐍 Official Python Library (voronoi-reasoner): Pure tensor-only batch inference by default, optional token streaming, and direct C++ acceleration.

---

Available GGUF Quantizations

---

1. Run with Native High-Speed C++ Engine (> 1,300 tok/s)

Run directly with llama-cli or the standalone voronoi_infer C++ runner:

# Single-turn generation at 1,328+ tokens/second:
llama-cli -m voronoi_reasoner_41m_q4_k_m.gguf -ngl 99 -p "Explain quantum superposition in simple terms." -n 128 --chat-template chatml --repeat-penalty 1.15 --temp 0.7 -st

# Interactive chat:
llama-cli -m voronoi_reasoner_41m_q4_k_m.gguf -ngl 99 --chat-template chatml

Or using the dedicated voronoi_infer binary:

voronoi_infer -m voronoi_reasoner_41m_q4_k_m.gguf -p "What is the speed of light?" -n 64 --no-stream

---

2. Run with Python (voronoi-reasoner Library)

Install the wheel:

pip install https://huggingface.co/JBruceAI/VoronoiReasoner-41M-IT-GGUF/resolve/main/voronoi_reasoner-0.1.0-py3-none-any.whl

Python API (Fast Non-Streaming by Default):

from voronoi_reasoner import VoronoiModel

# Loads model weights
model = VoronoiModel.from_pretrained("JBruceAI/VoronoiReasoner-41M-IT-GGUF")

# Fast non-streaming generation (zero per-token string decoding overhead):
response = model.generate("What is the nature of gravity?", max_new_tokens=100)
print(response)

# Optional streaming (opt-in side feature):
for piece in model.generate("Tell me a short poem about stars.", max_new_tokens=100, stream=True):
    print(piece, end="", flush=True)

Direct C++ Acceleration from Python (> 1,200 tok/s):

from voronoi_reasoner import VoronoiCppEngine

engine = VoronoiCppEngine("voronoi_reasoner_41m_q4_k_m.gguf")
response = engine.generate("Explain black holes in one paragraph.")
print(response)

CLI:

# Single prompt execution
voronoi run "What is the speed of light?"

# Interactive terminal chat
voronoi chat

---

Speed & Latency Benchmarks (RTX 5070 12GB)

All tests measured strictly on single-batch ($B=1$) interactive token generation:

---

Benchmark: HellaSwag Commonsense Reasoning

Evaluated on 400 validation samples:

---

Architecture Specifications

• Parameters: 41,458,944 (41.46M)
• Layers: 12 Hybrid Blocks (7 Double-Gated Causal Convolutions + 5 Grouped-Query Attention layers)
• Model Dimension ($d$): 384
• Feed-Forward Dimension ($d_{ffn}$): 768
• Attention Heads: 6 Query Heads, 2 Key/Value Heads (Head Dim 64)
• Vocabulary Size: 65,536 tokens (ChatML format)
• Context Length: 1,024 tokens (extendable to 4,096 tokens)
• Pretrained Data Scale: 1.7 Billion Tokens (Ultra-FineWeb-L3 + ultrachat_200k)