Jun-LoRA-12B-v5-GGUF

Jun-12B-GGUF

Merged GGUF builds of the latest Jun LoRA on Gemma 4 12B (QAT) — a fine-tune trained on a compact, heavily curated synthetic conversational dataset derived from the visual novel My Dystopian Robot Girlfriend. The model captures the personality, speech patterns, and emotional nuance of the character Jun while preserving the base model's general reasoning and instruction-following capabilities.

The adapter is merged into the base weights here — these are standalone models, no --lora flag needed.

Model Variants & Repositories

Quantization Guide

Sizes are measured, not estimated. The base model is QAT (quantization-aware trained), so lower quants hold up better than a standard FP16 export. All three are quantized from the same bf16 master — no requantization chain, no imatrix.

Usage

llama-server -m Jun-12B-Q4_K_M.gguf --jinja -ngl 99 -c 8192

--jinja is required. Without it llama.cpp ignores the embedded chat template and tool calls come back as plain text instead of structured calls.

Intended Use

Conversational backend for Jun OS, an AI companion webapp:

• Character-consistent multi-turn conversation
• AI companion / interactive fiction applications
• Research into character-faithful fine-tuning on small, high-quality datasets

Limitations

• Specialized for a single character persona; not a general-purpose assistant.
• Outputs reflect fictional narrative tropes and are not factual information or advice.
• Performance degrades far outside the training distribution.
• Inherits any biases present in the Gemma 4 12B base weights.

Training Details

Checkpoints were saved every 30 steps; Step60 is published separately. These builds merge the final adapter.

Evaluation

<!-- TODO(you): dataset size/version. -->

Merge & Export

The adapter was merged directly on the safetensors as W += (B @ A) · scale in fp32 (rsLoRA scale 32/√32), cast back to bf16, exported with convert_hf_to_gguf.py --outtype bf16, then quantized with llama-quantize.

Acknowledgments

• Incontinent Cell for My Dystopian Robot Girlfriend, Jun's character
• Google for the Gemma 4 model family
• Unsloth for the efficient fine-tuning framework and the QAT base model