chyon_gemma_4_gguf_v2

Chyon Gemma 4 GGUF

Verified Q4_K_M and Q5_K_M GGUF builds of the AI 키온이 (Chyoni) conversational model for local inference with llama.cpp-compatible runtimes.

Chyoni is a Korean-first educational mentor focused on gifted education in the UK, mathematical and scientific reasoning, practical applications of AI, educational content, and solopreneur strategy.

Key facts

The verified Unsloth 16-bit merge is the authoritative conversion source. The earlier standard PEFT merge was rejected because the resulting model repeatedly emitted prompt-template tokens.

Files

Prompt format

The recorded GGUF verification placed the complete persona prompt and the question in one user turn:

<start_of_turn>user
{{ persona_prompt }}

{{ question }}<end_of_turn>
<start_of_turn>model

Use the full Korean or English persona prompt from the LoRA model card. Recommended stop tokens are <end_of_turn>, <turn|>, and <eos>.

Run with llama.cpp

Save the formatted prompt as prompt.txt, then run:

llama-cli \
  -m chyon-gemma-4-v2.Q4_K_M.gguf \
  -f prompt.txt \
  -n 300 \
  --temp 0

Deterministic decoding (--temp 0) reproduces the recorded verification path.

Verification

Conversion used llama.cpp's convert_hf_to_gguf.py with an F16 intermediate, followed by llama-quantize for Q4_K_M and Q5_K_M. These are narrow artifact-integrity checks, not standardized quality or safety benchmarks.

Intended use

• Local Korean conversational-assistant experiments
• Educational and solopreneur persona demonstrations
• Testing the verified Chyon merge in llama.cpp-compatible applications

Limitations

• The source adapter was trained on a small, creator-curated, partly template-augmented dataset.
• Training showed a large loss gap: 0.8608 training versus 4.7604 validation.
• No aggregate test result or standardized factuality, safety, bias, or robustness benchmark is published.
• English and multimodal behavior were not separately validated.
• Quantization may alter output quality relative to the merged model.
• Generated education, business, mathematics, or science claims may be inaccurate or overconfident.

Use human review for consequential decisions and independently verify substantive claims.