Downloads · 30 days
4
3% of all-time downloads
johnhalloran/goramen-blog-gemma-4-sft-gguf
goramen-blog-gemma-4-sft-gguf is a machine learning model from johnhalloran. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as gemma.
GGUF conversion of johnhalloran/goramen-blog-gemma-4-sft (the bf16 fused checkpoint), for CPU/llama.cpp inference off Apple Silicon.
Downloads · 30 days
4
3% of all-time downloads
All-time downloads
120
Public
Repo size
5.3 GB
Likes
0
Public
Click a slice to open those files.
.gguf5.3 GB · 100%
From the Hugging Face model README
GGUF conversion of johnhalloran/goramen-blog-gemma-4-sft
(the bf16 fused checkpoint), for CPU/llama.cpp inference off Apple Silicon.
Converted with llama.cpp's convert_hf_to_gguf.py (Gemma4ForConditionalGeneration →
Gemma4Model text path), then quantized to Q4_K_M with llama-quantize. Text-only —
the source checkpoint has no vision tower weights (see the base model's card).
| File | Quant | Size |
|---|---|---|
goramen-blog-gemma-4-sft-Q4_K_M.gguf | Q4_K_M | ~4.9 GB |
pip install llama-cpp-python
python -c "
from llama_cpp import Llama
llm = Llama.from_pretrained(
repo_id='johnhalloran/goramen-blog-gemma-4-sft-gguf',
filename='*Q4_K_M.gguf',
)
print(llm.create_chat_completion(messages=[
{'role': 'system', 'content': \"You are the author of Go Ramen, a ramen review blog. Write in the blog's distinctive, casual, food-obsessed voice.\"},
{'role': 'user', 'content': 'Write a Go Ramen blog post titled \"Afuri - Portland, OR\". Labels: OR-Portland.'},
])['choices'][0]['message']['content'])
"
Used by the johnhalloran/goramen-blog-demo Space.