Spark-X2.5-4B-GGUF

Spark-X2.5-4B GGUF — experimental v0.1

GGUF quantizations of XHToken/Spark-X2.5-4B, pinned to source revision 1e4c2477f330d1f11304d0dc092a9dbf4075f470.

This is an experimental v0.1 release built with Spark2.5 support from llama.cpp commit fe158c6c4db8b0cb8d74c7cfe23401f7c21a45fe (build 10668). Use a llama.cpp build containing compatible spark2_5 support.

Files

The published Q6_K is the mixed-context iMatrix candidate. Compared with the 4K-only iMatrix candidate, it was directionally better on most 16K metrics, but the difference is small and the quick-run uncertainty overlaps. The clear result is that either iMatrix build substantially improves Q6_K over the no-iMatrix ablation.

Quick use

llama-cli.exe `
  -m .\Spark-X2.5-4B-Q6_K.gguf `
  -ngl 99 `
  -fa on `
  -c 16384

This llama.cpp build selects the embedded chat template without the obsolete -cnv flag. Reduce -ngl if another application is using VRAM. The 16K passkey checks below used -ngl 28 so the same compute path could also be used for the BF16 reference.

Fidelity results

All candidates were compared token-by-token against the same converted BF16 GGUF. Lower KLD/RMS is better; higher Same top p is better. “Same top p” is llama.cpp's name for top-1 agreement, not nucleus-sampling top-p.

Neutral held-out evaluation at 4K

Protocol: AtomicChat eval_neutral, 2 chunks, 4,096 scored positions, context 4,096, -ngl 28, batch 2,048, micro-batch 512, 8 threads and Flash Attention enabled. The BF16 KLD base is 1,073,283,076 bytes with SHA256 712B7248BCC72D0057EC56F61AF838D251A83EDA3F938DDD5E54FA03BFFCE322.

The same evaluation at 16K

Protocol: the same held-out corpus and compute path, 2 chunks, 16,384 scored positions, context 16,384. The BF16 KLD base is 4,294,705,156 bytes with SHA256 CB3BDBC2088CD3399ABCFD5D35B427B9F5C0463EFBF8FE4C9F1D68B28804AB52.

These are deliberately labelled quick2 measurements. They are useful for selecting a v0.1 candidate, not a substitute for a full benchmark suite.

iMatrix and calibration

Q6_K was quantized directly from the BF16 GGUF; it was never requantized from Q8_0.

• BF16 reference SHA256: 8CECF405A41A4A10F833530910C2E13FDE9FB39C325C8AFC3C5D10E4181E1A14
• iMatrix SHA256: 696688C6E4CB5AD6B1A2F4E3F460BBDCA50C2AC3CDBB7BC13C0455CA9F4BE489
• Main calibration: 2,022,075 Spark tokens, context 4,096
• Long-context calibration: 279,736 Spark tokens, context 16,384
• Observations merged per tensor: 2,019,328 from the 4K pass and 278,528 from the 16K pass, approximately 88% / 12%
• Batch size: 2,048; --parse-special enabled
• Calibration corpus SHA256: DD4DADD436A3AB9329C2EE98000CC2FF6540E9C190400292A99EB736AA9E8796
• Long-context corpus SHA256: 219DF033B7F9B30787C6DEC452FA16967A7C6FD4C0F2E42D84282E26C2AF940B
• Corpus source: AtomicChat calib-corpora commit 814d662f6c94d207fc8f38545a1b4abea11484b5

Calibration and evaluation inputs are disjoint. The calibration manifests and the 3.4 MiB mixed-context iMatrix are included under artifacts/; the full text corpora are not redistributed in this model repository.

Reproduction of the selected quantization:

llama-quantize.exe `
  --imatrix .\artifacts\imatrix\Spark-X2.5-4B-imatrix-4k-16k.gguf `
  .\Spark-X2.5-4B-BF16.gguf `
  .\Spark-X2.5-4B-Q6_K-raw.gguf `
  Q6_K

python <LLAMA_CPP>\gguf-py\gguf\scripts\gguf_new_metadata.py `
  .\Spark-X2.5-4B-Q6_K-raw.gguf `
  .\Spark-X2.5-4B-Q6_K-reproduced.gguf `
  --remove-metadata quantize.imatrix.file `
  --force

The BF16 input is intentionally not included in this repository. Convert the pinned upstream revision with the same llama.cpp commit before reproducing the quantization.

The raw evaluated Q6_K container had SHA256 162985111D4DD15DA0FF2AD111965CABCF60690C40A52D53D2CB3640483DE51D. Before publication, the non-functional quantize.imatrix.file metadata key was removed because llama.cpp had stored its local absolute path. gguf_hash.py --no-layer gives the same tensor-data SHA256 before and after sanitization: f70482cf653bc90d935ab68ab83202d2a02c260699c5213d188d736c469bd1ae. The published container therefore has exactly the evaluated weights despite its different whole-file hash.

16K functional check

A deterministic synthetic passkey was inserted near 10%, 50% and 90% of a 14,997-token prompt. BF16, Q8_0, Q6_K without iMatrix, Q6_K with 4K iMatrix and Q6_K with mixed 4K+16K iMatrix all passed exact-match retrieval at all three depths: 15/15 runs.

This test is saturated and should only be read as a functional 16K smoke test. It does not distinguish the quantizations and does not establish general long-horizon reasoning quality. Prompts, manifests and result JSON files are included under artifacts/passkey_16k/.

RTX 4060 Laptop observations

llama-bench, CUDA, full GPU offload (-ngl 99), Flash Attention enabled:

At context 16K, the reported VRAM figures were approximately 4.5 GB total for Q8_0 and 3.5 GB total for Q6_K. They include roughly 0.4–0.5 GB already allocated to the Windows/display system because this laptop setup has no iGPU handling the display. The model-run increments were therefore approximately 4.0–4.1 GB for Q8_0 and 3.0–3.1 GB for Q6_K. These were manual observations; treat VRAM and speed values as hardware- and runtime-dependent, not universal requirements.

Reproducibility artifacts

The artifacts/ directory contains:

• parsed metrics in results.json;
• source, model, iMatrix, evaluation-base and release hashes in audit-manifest.json and SHA256SUMS.txt;
• raw KLD, quantization, iMatrix, tokenizer and speed logs;
• the Q6_K public-container metadata sanitization and tensor-identity proof;
• calibration manifests;
• the exact benchmark and quantization scripts used locally (paths must be adapted before reuse);
• the 16K passkey prompts and result JSON files.

The approximately 1 GiB and 4 GiB KLD base files are not uploaded in v0.1. Their SHA256 values, exact protocols and raw comparison logs are preserved so they can be regenerated and checked.

Next steps

The next experimental phase will explore mixed-context iMatrices more systematically. In particular:

• vary the share of 4K and long-context calibration observations instead of testing only the current approximately 88% / 12% mix;
• test additional long-context calibration lengths and task families, rather than relying on a single 16K corpus recipe;
• evaluate each recipe at matching long contexts with the same BF16 reference and compute path;
• retain a new recipe only when the improvement is repeatable across KLD, tail KLD and functional long-context checks.

The present 4K+16K result is therefore a useful first data point, not a claim that this mixture is optimal.

Validation status and limitations

• HF/llama.cpp tokenizer parity passed 10/10 fixed cases, including multilingual text and special-token/chat-template cases.
• GGUF architecture, tokenizer metadata and embedded chat template checks passed.
• A direct Transformers-vs-GGUF logit parity comparison was not completed for v0.1. Consequently, KLD results establish fidelity to the converted BF16 GGUF; they do not independently prove that the experimental conversion is numerically identical to the upstream Transformers implementation.
• Q8_0 has strong local provenance: the quantization command was recovered and metadata match the current BF16. However, the BF16 hash was not recorded at the exact time of the original Q8_0 run, so this is not a complete cryptographic chain of custody.
• Spark2.5 support was experimental at the pinned llama.cpp commit. Compatibility with older releases is not expected.
• No benchmark here validates the upstream maximum-context claim. Only 4K distributional checks and a 16K distributional/functional check were run.

See the upstream model card for architecture, intended use and model-level limitations. The upstream Apache-2.0 license is included as LICENSE.