Spark-x2.5-4B-Heretic-jp

Spark-X2.5-4B Heretic JP (full weights)

Full-precision (BF16) Japanese-adapted, refusal-attenuated derivative of the XHToken/Spark-X2.5-4B model.

This is the source weight for the GGUF release at soyaakinohara/Spark-X2.5-4B-Heretic-jp-gguf. The uncensored-only weight (before Japanese adaptation) is published separately at soyaakinohara/Spark-X2.5-4B-Heretic.

---

Files

model.safetensors          # 7.7 GiB, single shard, BF16
config.json
configuration_spark.py     # custom model code (trust_remote_code=True required)
modeling_spark.py
tokenizer.json / tokenizer_config.json
chat_template.jinja
generation_config.json

---

transformers usage

Custom model code is required:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained(
    "soyaakinohara/Spark-x2.5-4B-Heretic-jp", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "soyaakinohara/Spark-x2.5-4B-Heretic-jp",
    dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)

msgs = [{"role": "user", "content": "日本の首都は?"}]
text = tok.apply_chat_template(msgs, tokenize=False,
                               add_generation_prompt=True,
                               enable_thinking=False)
inp = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inp, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][inp["input_ids"].shape[1]:], skip_special_tokens=True))

Notes:

• This is a thinking model. Without enable_thinking=False, responses start with a thinking trace. Allow 1024 or more new tokens when thinking is on.
• For tool use, pass tools=[{"type": "function", "function": {...}}] to apply_chat_template. Assistant tool calls render as <tool_call> blocks.

---

Reported local performance

The model was prepared and tested on:

OS:       Ubuntu 24.04
GPUs:     NVIDIA GeForce RTX 5060 Ti 16GB x 2
RAM:      32GB + 23GB swap
Runtime:  PyTorch (BF16) + transformers

Training ran in BF16 + LoRA (r16) with a 2-GPU layer split.

---

Provenance

Base model

• XHToken/Spark-X2.5-4B
• Base revision used locally: 1e4c2477f330d1f11304d0dc092a9dbf4075f470
• License: Apache-2.0

Adaptation steps

1. Refusal attenuation with Heretic (refusals 58→3/100, KL 0.0118).
2. Japanese continued pretraining on 50k Japanese Wikipedia articles (train loss 2.77→2.20).
3. Japanese instruction tuning on 50k rows (train loss 2.59→1.50).
4. Tool-calling repair on 8k tool-use rows (train loss 1.48→0.57). After this step, complex tool definitions fire correctly (function-name match 96% on a held-out Japanese tool-call set).

Datasets

• fn-aka-mur/wiki40b_ja (continued pretraining)
• izumi-lab/llm-japanese-dataset-vanilla (instruction tuning, CC-BY family)
• nappa0326/glaive-function-calling-v2-sharegpt-japanese (tool repair, MIT)
• Tonari-no-usagi/Japanese_Function_Calling_Dataset (evaluation only; CC-BY-NC, not used for training)

---

AI assistance disclosure

The local model preparation workflow, training, validation, and this model card were performed with assistance from Hermes Agent. The model was then reviewed and published by the repository owner.

---

Responsible use

This is an uncensored / refusal-removed model. It may produce content that an aligned model would refuse, including unsafe, illegal, or harmful material. It has no reliable built-in safety layer. Use appropriate access controls, moderation, logging, and human review for any deployment, and comply with all applicable laws and policies.

The model is provided as-is. Users are responsible for prompts, outputs, and any downstream actions based on them.

---

日本語

概要

XHToken/Spark-X2.5-4B をベースに、無検閲化と日本語適応を行ったモデルのフル重み (BF16) です。 GGUF 版の変換元でもあります。

無検閲化には Heretic による refusal 減衰を使い、 日本語 Wikipedia での継続事前学習、日本語指示データでの調整、 ツール呼び出しの追加学習を行っています。

trust_remote_code=True が必要です。