Spark-X2.5-4B-GGUF

Spark-X2.5-4B GGUF

This repository contains GGUF format quants for XHToken/Spark-X2.5-4B.

Spark-X2.5-4B is a compact, high-efficiency language model featuring a hybrid attention architecture (combining sliding-window attention with full attention layers) and native context lengths up to 1M tokens.

These GGUF files were converted from the original safetensors weights using a custom Spark 2.5 build of llama.cpp at BF16 precision, then quantized into standard k-quants.

---

Available Files and Quantizations

---

Requirements

Spark-X2.5 uses the Spark2_5ForCausalLM (spark2_5) architecture. To run these files, ensure your local llama.cpp installation or downstream GUI client includes Spark 2.5 runtime support.

---

How to Use

1. With llama.cpp

Run the model directly from the command line using llama-cli:

llama-cli -m ./Spark-X2.5-4B-Q4_K_M.gguf \
    -p "You are a helpful and concise assistant.\nUser: Hello! What can you do?\nAssistant:" \
    -n 512 \
    -c 4096 \
    --temp 0.7

To run as a local server:

llama-server -m ./Spark-X2.5-4B-Q4_K_M.gguf \
    --host 127.0.0.1 \
    --port 8080 \
    -c 4096

2. With Ollama

Create a Modelfile in the same directory:

FROM ./Spark-X2.5-4B-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.7

Then build and run the model:

ollama create spark-x2.5-4b -f Modelfile
ollama run spark-x2.5-4b

3. With LM Studio

1. Download or copy any .gguf file to your LM Studio models folder.
2. Search for Spark-X2.5-4B in your local models tab.
3. Load the model with default GPU offloading settings and chat.

---

Original Model Details

• Original weights: XHToken/Spark-X2.5-4B
• Base Architecture: Spark 2.5 (Spark2_5ForCausalLM)
• Context Length: Up to 1,048,576 tokens (1M)
• Vocabulary Size: 131,072 tokens
• License: Apache 2.0

---

Attribution & Credits

Original model weights and architecture developed by XHToken.

GGUF conversions produced by Pink Pixel.

Made with 💖 by Pink Pixel