Maple-Preview-Exact-GGUF

Maple-Preview — Exact GGUF Repacks

DeepGrove · 2026
Community GGUF repacks, verification and NVIDIA Tesla V100 benchmarks by Arx12.

This repository contains alternative GGUF representations derived from the official DeepGrove Maple-Preview release and its official GGUF conversion:

deepgrove/maple-preview-GGUF

The files in this repository are derived from:

maple-preview-TQ2_0-head-F16.gguf

[!NOTE] This is a community derivative and is not an official DeepGrove release.

“Exact” refers specifically to the conversion of the source TQ2_0 matrix tensors into Q4_0/Q8_0 blocks while preserving their represented dequantized values. It does not mean that the model's pre-TQ2 training precision has been recovered.

About Maple-Preview

Maple-Preview is an open-source 20B-A1B ternary-weight reasoning model. DeepGrove positions the preview primarily as a reasoning-focused model, with strong performance for its memory footprint and support for very long context.

Key model characteristics:

• 20B total parameters / approximately 1B active parameters
• 24 transformer layers
• 256 experts, 8 active per token
• 131,072-token context
• ternary-weight MoE design
• MIT license

The upstream model card notes that this preview focuses primarily on raw reasoning. Agentic behavior and broad post-training are still limited compared with the intended full Maple release.

Architecture

Maple-Preview uses a 24-layer Mixture-of-Experts architecture with 256 experts and 8 active experts per token.

The model uses a 3:1 SWA-512:GA attention configuration, combining sliding window attention with global attention for long-context operation.

Upstream evaluation notes

DeepGrove describes Maple-Preview as targeting a strong memory-to-performance and speed-to-performance tradeoff for its weight class.

The upstream evaluation focuses on reasoning-oriented benchmarks including LCBv6, AIME 2026, HMMT 2026 and GPQA-D, using the dense output head.

Because this is a preview release, DeepGrove notes that agentic-task performance may lag behind raw reasoning performance and that broader post-training is still planned.

Exact GGUF repacks

Exact TQ2_0 repack

The source GGUF contains 168 TQ2_0 matrix tensors.

For the Q4_0 Exact and Q8_0 Exact variants, these tensors were not dequantized to floating point and then requantized with newly calculated scales.

Instead, each source TQ2_0 block is directly repacked while preserving the original FP16 block scale bit-for-bit.

Source TQ2_0 values

code 0 -> -d
code 1 ->  0
code 2 -> +d
code 3 -> +2d

Exact Q4_0 representation

7  -> -d
8  ->  0
9  -> +d
10 -> +2d

Exact Q8_0 representation

-1 -> -d
 0 ->  0
+1 -> +d
+2 -> +2d

The original FP16 scale d is copied into the destination blocks.

Runtime precision note

The represented matrix values are preserved by the exact repack, but runtime logits and generated text are not guaranteed to be bit-identical across TQ2_0, Q4_0 and Q8_0 kernels because different kernels may accumulate operations differently.

Verification

A block-level verifier checked all source TQ2_0 matrix tensors against both the Q4_0 and Q8_0 outputs.

TQ2 tensors: 168

[ 24/168] verified
[ 48/168] verified
[ 72/168] verified
[ 96/168] verified
[120/168] verified
[144/168] verified
[168/168] verified

All 168 / 168 source TQ2_0 matrix tensors passed.

The verifier checks:

• destination tensor type
• source and destination block counts
• FP16 scale bytes
• every packed Q4_0 code
• every Q8_0 integer code

Available files

token_embd.weight remains F16 in the Q4/Q8 release variants.

Precision notes

Q4_0 Exact + Q4_K head

Maple-Preview-Q4_0-Exact-head-Q4_K.gguf uses the exact/value-preserving TQ2_0 -> Q4_0 repack for the 168 source matrix tensors.

However, output.weight is conventionally quantized from F16 to Q4_K.

Therefore:

• matrix body: exact/value-preserving repack
• LM head: normal lossy Q4_K quantization
• token embedding: F16

F16 Expanded from TQ2

Maple-Preview-F16-Expanded-from-TQ2.gguf expands the values already represented by the source TQ2_0 tensors into F16 storage.

It is not an original pre-quantization FP16 checkpoint and cannot restore information that was not present in the source TQ2_0 representation.

NVIDIA Tesla V100 benchmark

Test system

GPU:          1x NVIDIA Tesla V100-SXM2-16GB
Driver:       580.173.02
CUDA:         13.0

CPU:          2x Intel Xeon E5-2670 v3
RAM:          31.25 GiB
OS:           Ubuntu 24.04

llama.cpp:    DeepGrove Maple fork
commit:       8ce8ca6c6d370b6235dfa8e2a0611a9adb6d77d1
CUDA arch:    sm_70

Controlled benchmark configuration

Context:                 131072
Parallel:                1
Device:                  CUDA0
KV cache K/V:            Q8_0
KV offload:              enabled
Op offload:              enabled
Temperature:             0
GGML_CUDA_FORCE_MMQ:     1

Prompt tokens:           36
Generated tokens:        1500

Prompt:

Write a detailed explanation of how a modern CPU works, including caches, branch prediction, pipelines, memory hierarchy and multithreading.

For models that fit in VRAM, all model layers were offloaded to the V100.

For Q8_0 Exact and F16 Expanded, --fit on was used so llama.cpp could place as much of the model as possible on the 16 GiB V100 and keep the remaining tensors on the host.

Performance comparison with the original GGUF

Main result

On this Tesla V100 system, Maple-Preview-Q4_0-Exact-head-Q4_K.gguf increased:

• prompt processing from 98.63 t/s to 343.11 t/s (3.48x)
• decode from 39.09 t/s to 235.83 t/s (6.03x)

The Q4_0 Exact + F16-head variant reached 210.66 t/s decode, or 5.39x the original, while keeping the original F16 LM head.

The speedup comes from the physical representation and kernel path. The exact Q4_0/Q8_0 body repacks do not add information to the source TQ2_0 weights.

Relative size and behavior

VRAM / host placement observations

Q4_0 Exact + F16 head

llama-server VRAM: ~12,750 MiB
Placement:         Full GPU

Q4_0 Exact + Q4_K head

llama-server VRAM: ~12,324 MiB
Placement:         Full GPU

Q8_0 Exact + F16 head

llama-server VRAM: ~14,778 MiB
Total GPU usage:   ~15,089 MiB including ~308 MiB used by another process
Placement:         Hybrid GPU + CPU

F16 Expanded from TQ2

llama-server VRAM: ~14,794 MiB
Total GPU usage:   ~15,105 MiB including ~308 MiB used by another process
Container RSS:     ~20.67 GiB
Placement:         Hybrid GPU + CPU

For mmap-backed models, container RSS alone should not be interpreted as the total host memory footprint because mapped model pages may also be accounted for through the operating system page cache.

Q8_0 mmap A/B test

The Q8_0 Exact hybrid configuration was also tested with and without mmap.

--no-mmap improved prompt processing substantially in this test, but mmap enabled produced slightly better decode throughput and total benchmark time.

F16 Expanded result

The F16-expanded representation is intentionally included as a reference/debug artifact rather than a recommended inference format.

On the single 16 GiB V100 system it required hybrid GPU+CPU execution:

Prompt processing: 0.94 t/s
Decode:             7.35 t/s
Total time:         242.35 s / 1536 tokens

Because it contains no additional source information compared with the original TQ2_0 matrix weights, its much larger storage footprint should not be interpreted as higher model quality.

Runtime

Runtime compatibility

These GGUF files were produced and tested with DeepGrove's Maple-enabled llama.cpp fork:

https://github.com/deepgrove-ai/llama.cpp

Tested commit:

8ce8ca6c6d370b6235dfa8e2a0611a9adb6d77d1

At the time of these tests, the stock ggml-org llama.cpp build used for comparison did not recognize the maple architecture.

Use the DeepGrove fork unless Maple support has since been upstreamed.

Example: CUDA / Tesla V100

GGML_CUDA_FORCE_MMQ=1 ./llama-server \
  -m Maple-Preview-Q4_0-Exact-head-Q4_K.gguf \
  --ctx-size 131072 \
  --parallel 1 \
  --device CUDA0 \
  --gpu-layers all \
  --split-mode none \
  --main-gpu 0 \
  --fit off \
  --no-host \
  --kv-offload \
  --op-offload \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --jinja

Reproducibility

The quantizer modification used for the exact TQ2_0 -> Q4_0/Q8_0 repack is included as:

exact-tq2-repack.patch

SHA256 hashes are included in:

SHA256SUMS

Current release hashes:

80b74240328aee4e3d3f708bfe2cd02bf28c03278f08a83c1dd28c260f834e97  Maple-Preview-Q4_0-Exact-head-Q4_K.gguf
10be6a9dd28cbfcb414009f31cbfd2126c245a81ac05062cd5a5e2193ce732ac  Maple-Preview-Q4_0-Exact-head-F16.gguf
f79c20a931b096717eb0dda844c7b4e33230ffa7dfddb45015ec8151d24a6ce7  Maple-Preview-Q8_0-Exact-head-F16.gguf
8b4da7b2fdc38a9af9e76d090fae22e8f784534f4362782ee7e5867c0d4a3a79  Maple-Preview-F16-Expanded-from-TQ2.gguf

Upstream

• Base model: https://huggingface.co/deepgrove/maple-preview
• Official GGUFs: https://huggingface.co/deepgrove/maple-preview-GGUF
• Maple-enabled llama.cpp: https://github.com/deepgrove-ai/llama.cpp

This repository contains derived/community GGUF representations and is not the upstream Maple release.

Limitations

The upstream Maple-Preview model card describes this release as having limited post-training for agentic tasks and relatively small-scale general reinforcement learning. These GGUF repacks do not change those model-level limitations.

The exact repacks also do not restore information that was absent from the source TQ2_0 representation.

License

Maple-Preview is released under the MIT License. Refer to the upstream DeepGrove repository for the original model, license and attribution.