goramen-blog-gemma-4-sft

goramen-blog-gemma-4-sft

A QLoRA fine-tune of mlx-community/gemma-4-e4b-it-4bit (itself an MLX conversion of google/gemma-4-E4B-it) trained on the full text archive of the Go Ramen blog. Given a post title and labels, it writes a ramen review in the blog author's own voice.

from mlx_lm import load, generate

model, tokenizer = load("johnhalloran/goramen-blog-gemma-4-sft")

prompt = 'Write a Go Ramen blog post titled "Afuri - Portland, OR". Labels: OR-Portland.'
messages = [
    {"role": "system", "content": "You are the author of Go Ramen, a ramen review blog. "
                                   "Write in the blog's distinctive, casual, food-obsessed voice."},
    {"role": "user", "content": prompt},
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_dict=False)

response = generate(model, tokenizer, prompt=prompt, max_tokens=400, verbose=True)

pip install mlx-lm
python -m mlx_lm generate --model johnhalloran/goramen-blog-gemma-4-sft \
  --prompt 'Write a Go Ramen blog post titled "Afuri - Portland, OR". Labels: OR-Portland.' \
  --max-tokens 400

About this model

Precision: bf16, not quantized. The QLoRA adapter was trained against a 4-bit quantized base, but the weights published here were dequantized to bf16 before fusing, and the LoRA delta was added in that dense bf16 space — not fused directly into the still-quantized weights and requantized afterward. That distinction matters: fusing into quantized weights means dequantize → add delta → requantize the sum, which throws away precision a second time on top of the base model's original quantization error. Fusing into the dequantized backbone instead adds the adapter cleanly, once, at full precision. If you need a smaller, quantized artifact, quantize this bf16 model yourself (mlx_lm.convert -q) rather than starting from a requantized fuse.

Text-only, despite a multimodal base. This checkpoint descends from a nominally multimodal (text + vision + audio) base model, but mlx_lm's gemma4 model class only ever instantiates the text decoder (language_model) — it doesn't load the base checkpoint's vision_tower or audio_tower weights at all, so a LoRA fine-tune run through mlx_lm.lora never sees the training examples' photos, and fusing the adapter back in through mlx_lm.fuse only ever had language_model.* weights to merge and save. Every tensor key in model.safetensors.index.json is prefixed language_model.* — no vision or audio weights are present. If you want an image-conditioned version of this fine-tune, it needs to go through mlx_vlm instead, which loads the vision tower for real; see the training repo for that (currently experimental) path.

Training details

Data

Trained on johnhalloran/goramen-blog: 1,841 posts scraped from the Go Ramen blog, split 1,567 train / 100 validation / 100 test (fixed seed, no overlap). Each example is a chat-format record — a system prompt establishing the blog's voice, a user turn (Write a Go Ramen blog post titled "..." Labels: ...), and the post body as the assistant completion, which is the only part trained on (prompt tokens are masked out of the loss).

Evaluation

Two checkpoints from the same run were scored on the held-out test split (100 posts, never seen during training or validation) to check for overfitting — evaluated against the 4-bit training checkpoints, before the dequantized fuse published here:

Train loss fell monotonically for the entire run (5.24 → 1.6), but validation loss bottomed out at iter 1410 and drifted upward for the remaining 70% of training — classic overfitting on a fairly small (1,567-example) dataset. This upload is fused from the iter-1410 checkpoint, not the final one, since it generalizes measurably better.

Limitations

• Style transfer, not fact-checking. The model learned the blog's voice and structure, not the truth about any specific ramen shop — don't treat generated posts as reliable information about a restaurant.
• Small fine-tuning set. 1,567 training examples is enough to shift tone and structure, not enough to fully avoid overfitting (see Evaluation above); expect some memorized phrasing from the training posts.
• English only, and specific to one blog's idiosyncratic voice — it will not generalize to other writing styles or topics.
• ~14GB download (bf16, 4 shards) — noticeably larger than a quantized model of the same parameter count. If size matters more than fuse precision for your use case, quantize this model yourself instead of using an earlier requantized-fuse artifact.
• Inherits the base model's general capabilities and limitations otherwise; see google/gemma-4-E4B-it for the base model card.

License / provenance

Released under the Gemma license, matching the base model. The training text and photos are the copyrighted work of the Go Ramen blog's author; the training dataset is a personal archival copy, not a license grant for reuse of that underlying content.