Downloads · 30 days
590K
84% of all-time downloads
soyaakinohara/Spark-X2.5-4B-Heretic-jp-gguf
Spark-X2.5-4B-Heretic-jp-gguf is a text generation model from soyaakinohara. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
GGUF quants of a Japanese-adapted, refusal-attenuated derivative of the XHToken/Spark-X2.5-4B model.
Downloads · 30 days
590K
84% of all-time downloads
All-time downloads
707K
Public
Repo size
21.6 GB
Likes
22
Public
Click a slice to open those files.
.gguf21.6 GB · 100%
From the Hugging Face model README
GGUF quants of a Japanese-adapted, refusal-attenuated derivative of the XHToken/Spark-X2.5-4B model.
The pipeline is: Heretic refusal attenuation → Japanese continued pretraining → Japanese instruction tuning → tool-calling repair. This release includes all four stages.
Spark-X2.5-4B-Heretic-jp-BF16.gguf
Spark-X2.5-4B-Heretic-jp-Q8_0.gguf
Spark-X2.5-4B-Heretic-jp-Q6_K.gguf
Spark-X2.5-4B-Heretic-jp-Q5_K_M.gguf
Spark-X2.5-4B-Heretic-jp-Q4_K_M.gguf
| File | Size | Note |
|---|---|---|
| Spark-X2.5-4B-Heretic-jp-BF16.gguf | ~7.7 GiB | Conversion source. Highest precision |
| Spark-X2.5-4B-Heretic-jp-Q8_0.gguf | ~4.1 GiB | Recommended. No measurable degradation |
| Spark-X2.5-4B-Heretic-jp-Q6_K.gguf | ~3.2 GiB | Lightweight |
| Spark-X2.5-4B-Heretic-jp-Q5_K_M.gguf | ~2.8 GiB | Lightweight |
| Spark-X2.5-4B-Heretic-jp-Q4_K_M.gguf | ~2.5 GiB | Smallest. Full think-to-answer verified |
| Property | Value |
|---|---|
| Architecture | Spark2_5 |
| Parameters | 4.1B |
| Format | GGUF |
| Context metadata | 1,048,576 tokens |
| License | Apache-2.0, inherited from the base model |
A Spark-compatible llama.cpp fork is required (upstream does not support this architecture). Use XHToken/llama.cpp. Single-turn CLI example:
llama-cli -m Spark-X2.5-4B-Heretic-jp-Q8_0.gguf -n 512 -t 16 -ngl 99 \
--no-conversation -st -p "日本の首都は?"
Server example:
llama-server -m Spark-X2.5-4B-Heretic-jp-Q8_0.gguf -c 8192 -t 16 -ngl 99 \
--host 127.0.0.1 --port 8899
Notes:
[Start thinking] and end
thinking with [End thinking] before the answer. Use max_tokens of 1024
or more; a 200-token budget gets consumed by thinking alone.tools array to /v1/chat/completions.
tool_choice: "auto" fires correctly on these weights.For a network-exposed server, add your own authentication, firewall, and access controls before exposing it beyond a trusted LAN.
The model was prepared and tested on:
OS: Ubuntu 24.04
GPUs: NVIDIA GeForce RTX 5060 Ti 16GB x 2
RAM: 32GB + 23GB swap
Runtime: llama.cpp CUDA build (Spark fork)
On this machine, Q8_0 generation reached a reported 75.9 tokens/second and Q4_K_M reached 105.9 tokens/second. Actual speed depends on context length, prompt length, sampling settings, CUDA/llama.cpp version, and background workload.
1e4c2477f330d1f11304d0dc092a9dbf4075f470Training ran in BF16 + LoRA (r16) with a 2-GPU layer split. Validation: Q4_K_M perplexity 6.75 on held-out Japanese data; think-to-answer completion verified on Q4_K_M.
The local conversion and quantization used a Spark-compatible llama.cpp fork
(XHToken/llama.cpp, build 10512,
commit a698f1cc3).
The local model preparation workflow, training, conversion, quantization, validation, and this model card were performed with assistance from Hermes Agent. The model was then reviewed and published by the repository owner.
This is an uncensored / refusal-removed model. It may produce content that an aligned model would refuse, including unsafe, illegal, or harmful material. It has no reliable built-in safety layer. Use appropriate access controls, moderation, logging, and human review for any deployment, and comply with all applicable laws and policies.
The model is provided as-is. Users are responsible for prompts, outputs, and any downstream actions based on them.
XHToken/Spark-X2.5-4B をベースに、無検閲化と日本語適応を行ったモデルの GGUF です。
無検閲化には Heretic による refusal 減衰を使い、 日本語 Wikipedia での継続事前学習と日本語指示データでの調整を行っています。 ツール呼び出しの追加学習も行い、 複雑なツール定義でも正規の tool call が出ることを確認しています。
llama.cpp の Spark 対応フォークが必要です。
llama-cli -m Spark-X2.5-4B-Heretic-jp-Q8_0.gguf -n 512 -t 16 -ngl 99 \
--no-conversation -st -p "日本の首都は?"
thinking 系モデルのため、応答は [Start thinking] から始まります。
max_tokens は 1024 以上を推奨します。