m51Lab-NorskGemma4-31B-GGUF

m51Lab-NorskGemma4-31B-GGUF

GGUF quantized versions of m51Lab-NorskGemma4-31B for local inference with llama.cpp, Ollama, LM Studio, and other GGUF-compatible tools.

NorEval score: 0.836 — Norway's top-scoring open-source model.

See full model card for complete benchmark details and training methodology.

Available Files

Usage

llama.cpp

# Download
huggingface-cli download dervig/m51Lab-NorskGemma4-31B-GGUF \
  NorskGemma4-31B-Q4_K_M.gguf --local-dir .

# Run (GPU accelerated)
./llama-cli -m NorskGemma4-31B-Q4_K_M.gguf \
  -p "Kva er hovudstaden i Noreg?" \
  -n 256 -ngl 99 -c 4096 --jinja

Ollama

# Create Modelfile
cat > Modelfile << 'EOF'
FROM ./NorskGemma4-31B-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER num_ctx 4096
SYSTEM Du er ein hjelpsom norsk AI-assistent.
EOF

# Create and run
ollama create norskgemma4 -f Modelfile
ollama run norskgemma4 "Forklar kva Stortinget er."

LM Studio

1. Download the Q4_K_M or Q8_0 file
2. Place in your LM Studio models directory
3. Select the model and start chatting in Norwegian

Performance

Tested with llama.cpp on 1x NVIDIA H100:

Credits

Built by m51.ai Lab. Based on Google Gemma 4 31B-it, fine-tuned with data from NbAiLab, evaluated on NorEval by LTG, University of Oslo.