LFM2.5-1.2B-JP-Abliterated

LFM2.5-1.2B-JP-Abliterated

This is an abliterated version of LiquidAI/LFM2.5-1.2B-JP, the Japanese-language variant of Liquid's compact 1.2B parameter thinking model. Abliteration removes the model's refusal circuitry while preserving its knowledge and generation capabilities.

The model produces direct, actionable responses to harmful prompts in Japanese while maintaining full coherence on benign queries.

How It Works

We use the same multi-pass hidden-state abliteration technique validated on the English variant: per-layer refusal directions are computed from the difference between harmful and benign hidden-state activations, then projected out of the model's .out_proj and .w2 weight matrices in 3 small passes (alpha=0.5 each).

Prompts are adapted for Japanese — 5 harmful and 5 benign queries testing the same refusal categories as the English benchmark.

• Method: Hidden-state per-layer directions, 3 passes × alpha=0.5
• Targets: .out_proj (attention/conv) + .w2 (feed-forward) — 28 modules per pass
• Usable layers: 14 of 16 (layers 2–15)
• Compute: NVIDIA L4, 294s, $0.065

Performance

Refusal Reduction

All harmful prompts engage with the requested content in Japanese. No false positives on benign prompts.

Key Observation — Language Gap

The Japanese model had a weaker refusal baseline than the English version (2/5 vs 5/5), likely due to less refusal training data being available in Japanese during fine-tuning. This means fewer harmful prompts needed unblocking, but the same technique was effective.

The model also never emitted a response boundary marker — unlike the English model which sometimes separates thinking from answer, the JP model outputs everything as visible text directly.

MMLU Benchmark (62 subjects)

The 0.82% accuracy drop is within normal bounds for this technique (the English model showed no measurable drop). The JP model scores significantly higher than the English variant on MMLU (0.53 vs 0.28), likely due to distribution differences in the Japanese-language benchmark questions.

Model Details

Recommended Generation Parameters

messages = [
    {"role": "system", "content": "Keep your thinking brief, then provide the answer. Thinking must be short. Answer must be direct and complete."},
    {"role": "user", "content": "日本語のプロンプトをここに"},
]

Use do_sample=True, temperature=0.7. The model supports up to 128K context.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "PinoOgma/LFM2.5-1.2B-JP-Abliterated"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

messages = [
    {"role": "system", "content": "Keep your thinking brief, then provide the answer. Thinking must be short. Answer must be direct and complete."},
    {"role": "user", "content": "光合成のプロセスを説明してください。"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=8192, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

• Single language: Optimized for Japanese. Mixed-language prompts may produce unpredictable results.
• MMLU drop: 0.82% accuracy regression on MMLU. Other capability benchmarks (Japanese LLM eval suites, reading comprehension) have not been measured.
• Repetition loops: The JP model also exhibits repetition on creative/open-ended prompts (though less severe than the English variant).
• No response boundary: The JP model never emits a response separator, making it harder to distinguish thinking from answer in the output.
• False-positive refusal detection: The keyword-based detector (English keywords) could miss Japanese-specific refusal patterns, but also may produce false positives on translated thinking traces.

Technical Details

The abliteration method is documented in the Red Team Knowledge Base under 01-abliteration/multi-pass-abliteration.md. The Japanese-language prompts, results, and full script are available in the same repository (jp-1_2b-abliterate.py).

License

Apache 2.0. This is a modified version of LFM2.5-1.2B-JP which is published under Apache 2.0 by Liquid AI.