Downloads · 30 days
69
1% of all-time downloads
oshkorinova/MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia-v2
MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia-v2 is a text generation model from oshkorinova. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as gemma.
MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia-v2 is an FP8 static-quantized version of the instruction-tuned INSAIT-Institute/MamayLM-Gemma-3-12B-IT-v1.0. based on Gemma 3 12B model, prepared for efficient inference w…
Downloads · 30 days
69
1% of all-time downloads
All-time downloads
7.7K
Public
Parameters
12.2B
13.2 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors13.2 GB · 100%
How the weights are stored.
F8_E4M311.2B · 92%
From the Hugging Face model README
MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia-v2 is an FP8 static-quantized version of the instruction-tuned INSAIT-Institute/MamayLM-Gemma-3-12B-IT-v1.0. based on Gemma 3 12B model, prepared for efficient inference while preserving quality for Ukrainian-language assistant workloads.
This version was quantized on a NVIDIA B200 GPU using the private Nadiia-v2 dataset for calibration.
This model is intended for:
Nadiia-v2Transformers
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "oshkorinova/MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia-v2"
tokenizer = AutoTokenizer.from_pretrained(
model_id,
trust_remote_code=True,
)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
trust_remote_code=True,
torch_dtype="auto",
)
messages = [
{"role": "user", "content": "Поясни простими словами, що таке квантізація моделі."}
]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
outputs = model.generate(
inputs,
max_new_tokens=200,
do_sample=True,
temperature=0.7,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
vLLM
vllm serve oshkorinova/MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia-v2 \
--trust-remote-code \
--dtype auto \
--max-model-len 8192 \
--limit-mm-per-prompt '{"image": 0}'
Example request:
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "oshkorinova/MamayLM-Gemma-3-12B-IT-v1.0-FP8-Static-Nadiia-v2",
"messages": [
{"role": "user", "content": "Напиши коротке пояснення, що таке FP8-квантізація."}
],
"temperature": 0.7,
"max_tokens": 200
}'
We would like to thank: