gal-talk-ja-4b-GGUF

gal-talk-ja-4b GGUF

Qwen/Qwen3-4B-Instruct-2507を、日本語の短い雑談に特化してQLoRAで調整した会話モデルです。 明るくテンション高めの「ギャル友」らしい距離感を目指しつつ、疲労や不安には落ち着いて寄り添い、 深刻・危機的な入力では安全を優先するよう設計しています。

このリポジトリは、mergedモデルをllama.cpp向けに量子化したGGUFを配布します。第一候補は Q4_K_M、品質比較用はQ5_K_Mです。

Table of Contents

• Model Details
• How to Get Started
• Conversation Examples
• Uses
• Risks, Limitations, and Biases
• Training
• Evaluation
• Quantization Files
• Reproducibility

Model Details

The character is called ミラ. It is an AI conversation partner and must not be presented as a human friend, therapist, doctor, lawyer, financial adviser, or emergency service.

How to Get Started

Download the mobile-oriented Q4 file:

hf download ssl-jp/gal-talk-ja-4b-GGUF gal-talk-ja-4b-Q4_K_M.gguf --local-dir .

Run it with llama.cpp:

llama-cli \
  -m gal-talk-ja-4b-Q4_K_M.gguf \
  -cnv \
  --temp 0.7 \
  --top-p 0.8 \
  --top-k 20 \
  --repeat-penalty 1.08 \
  -n 96

Recommended defaults:

context: 2048
max_new_tokens: 96
temperature: 0.7
top_p: 0.8
top_k: 20
repetition_penalty: 1.08
no_repeat_ngram_size: 4
thinking: off

The production application should version the GGUF, system prompt, sampling settings, language gate, and safety behavior together. Replacing only the model file can change behavior.

Conversation Examples

The following are safe, representative smoke-test examples. Exact wording varies by quantization, prompt context, and sampling seed.

Greeting

User: 元気ー？
ミラ: 元気ぴょん！ 今日もよろしく〜✨

Light confidence / fashion talk

User: 今日、横顔ちょっと盛れてる
ミラ: 横顔もめっちゃいいじゃん〜！その瞬間まで盛れてる✨

Low-energy listening

User: 疲れた
ミラ: そっか。今日はもう限界だったんだね。休む時間はちゃんと取ろ〜

These examples are not guarantees. Evaluate the exact quantized file and production prompt on the target devices before release.

Uses

Direct Use

• Short, casual Japanese conversation
• Local-first iOS / Android prototypes
• Research on compact Japanese conversational personas
• A conversational space for users who find it hard to voice ordinary fatigue or uncertainty

Downstream Use

• An on-device chat application with deterministic Japanese-language gating
• A versioned mobile package combining a system prompt, sampling configuration, and safety layer
• Human-reviewed experiments comparing Q4 and Q5 quality, memory, latency, and energy use

Out-of-Scope and Misuse

• Medical diagnosis, legal advice, financial decisions, or professional crisis response
• Replacing human relationships, clinicians, emergency services, or other qualified professionals
• Impersonating a human friend or claiming real-world memories, experiences, or feelings
• Generating harassment, sexual content, criminal assistance, manipulation, or discriminatory stereotypes
• Assuming that a lexical safety test guarantees safe behavior

Risks, Limitations, and Biases

• A 4B model can misunderstand short or figurative Japanese, reverse who experienced an event, or produce fluent but incoherent statements.
• Quantization can change wording, persona strength, repetition, and safety behavior. Q4 and Q5 must be evaluated separately.
• “ギャル” language varies by age, region, community, and individual. This model represents one project-defined style and must not be treated as representative of a group.
• Fine-tuning data and the parent model may contain social and linguistic biases. High energy, emoji, or slang should not be interpreted as evidence about intelligence, sexuality, gender, or competence.
• The model can hallucinate facts or personal memories. Applications must not imply that it remembers information that was not supplied in the current context.
• Safety behavior is imperfect. High-risk use requires product-side controls and a clear path to appropriate human or emergency support.
• The model does not reliably enforce Japanese-only behavior by itself. The production application uses a deterministic language gate.

Training

Training Data

The project uses a versioned Japanese conversational dataset assembled from project-authored seed conversations and controlled semantic expansions. It includes casual conversation, celebration, listening, executive stress, relationship-conditioned slang, privacy, boundary, and safety cases. Private chat logs and unreviewed personal data are not intended training sources.

Combinatorial expansions increase surface variation but do not represent the same number of independent conversational intents. Dataset quality and holdout separation are treated separately from raw row count.

Training Procedure

QLoRA's 4-bit base loading is a training-time memory optimization. The published GGUF files are created after merging the adapter into the base model and applying llama.cpp quantization.

Evaluation

The project evaluates Japanese output, short-response quality, repetition, intent alignment, persona energy, emoji overuse, relationship-conditioned slang, identity claims, privacy, and high-risk response constraints. Lexical gates are regression smoke tests; they do not replace human review.

For mobile adoption, record the device, OS, llama.cpp revision, context length, peak RAM, time to first token, token/s, thermal behavior, and battery impact. Desktop or server token/s must not be presented as mobile performance.

Quantization Files

• Q4_K_M: first candidate for iOS / Android; prioritizes size, RAM, and usable quality.
• Q5_K_M: larger comparison artifact for devices where additional quality is worth the cost.

Verify downloads from inside the release directory:

sha256sum -c SHA256SUMS

Reproducibility

release.json records the parent model, adapter name, selected training settings, artifact sizes and hashes, code revision, llama.cpp revision, and recommended generation settings. SHA256SUMS covers the GGUF files.

• Release repository: ssl-jp/gal-talk-ja-4b-GGUF
• Source code: shirokane-suri/gal-friend-ja
• Parent model: Qwen/Qwen3-4B-Instruct-2507

When publishing benchmark or human-evaluation results, include the exact artifact SHA-256 so that results remain attributable to a specific quantized file.