gemma-4-12b-soup-mvp

Gemma 4 12B Soup MVP — experimental pipeline proof

Not a production recipe assistant. No quality improvement is claimed. This release tests the engineering path: QLoRA optimizer updates, adapter reload, CPU merge, GGUF quantization and local Ollama generation. Tiny-data smoke tests are not benchmarks and cannot establish safety, generalization or superiority.

Fine-tuning and packaging: Eugene Evstafev — https://www.linkedin.com/in/eugene-evstafev/ . Base model: Google DeepMind, google/gemma-4-12B-it. Base revision: 707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7. Base and this release: Apache-2.0; see LICENSE and NOTICE. No endorsement by Google is implied.

Training scope

• Dataset: 3 training examples and 3 development-validation examples about soups.
• Synthetic source: gemini-3.1-flash-lite via LLM7, from the finetune-lab smoke dataset.
• Optimizer updates: 6; learning rate 0.0001; seed 42; microbatch 1.
• NF4 QLoRA with double quantization and BF16 compute.
• LoRA rank 8, alpha 16; language projection modules only.
• Prompt/header tokens masked. Non-thinking inference prefix used during training.
• No sealed holdout input used in training. No independent quality/safety benchmark claimed.
• This is a text-only adaptation. Multimodal quality was not evaluated; the GGUF is text-only.

Artifacts

• adapter/: PEFT adapter; exact base revision above is required.
• merged/: merged HF weights, tokenizer, processor and generation configuration.
• model-Q4_K_M.gguf: text-only Ollama deployment artifact.
• train.json, export.json, inference.json, dependencies.txt: observed run evidence.

GGUF SHA-256: 5db22e3f014721f87160599d947c09e8daf34ae0c8da909445b2488fa1bfbd34; bytes: 7381382848.

Hugging Face usage

from transformers import AutoTokenizer, Gemma4UnifiedForConditionalGeneration
repo = "EugeneEvstafev/gemma-4-12b-soup-mvp"
tokenizer = AutoTokenizer.from_pretrained(repo, subfolder="merged")
model = Gemma4UnifiedForConditionalGeneration.from_pretrained(
    repo, subfolder="merged", device_map="auto", dtype="auto")
text = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Suggest a simple lentil soup."}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tokenizer(text, add_special_tokens=False, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

Use the versions recorded in dependencies.txt (Transformers 5.10.1).

Ollama

ollama run chigwel/gemma-4-12b-soup-mvp

For local import, download the GGUF and use the included Modelfile with a local FROM path.

Limitations

Only a few examples and optimizer updates were used. Apparent improvements on these examples are not evidence of generalization. Do not rely on this model for allergen avoidance, dietary restrictions, cooking temperatures or food storage safety; verify such advice independently. It may hallucinate or regress from the base model. Quantization can change outputs. This is not a scientific paper.

Observed training trace

[
  {
    "gradient_norm": 3.39700984954834,
    "loss": 1.370896339416504,
    "step": 1
  },
  {
    "gradient_norm": 2.933994770050049,
    "loss": 1.7117650508880615,
    "step": 2
  },
  {
    "gradient_norm": 3.887418508529663,
    "loss": 1.9824351072311401,
    "step": 3
  },
  {
    "gradient_norm": 1.5510540008544922,
    "loss": 0.8900198340415955,
    "step": 4
  },
  {
    "gradient_norm": 1.9793601036071777,
    "loss": 1.0110812187194824,
    "step": 5
  },
  {
    "gradient_norm": 1.97877836227417,
    "loss": 1.3039186000823975,
    "step": 6
  }
]