mistral-nemo-humanities-gguf

Mistral Nemo 12B - Humanities Distilled (GGUF)

This model is a distilled version of Mistral Nemo 12B, fine-tuned on humanities content including:

• MMLU Humanities datasets (history, philosophy, religion, ethics)
• Synthetic data generated from GPT-OSS-120B

Training Details

• Base Model: mistralai/Mistral-Nemo-Instruct-2407
• Teacher Model: openai/gpt-oss-120b
• Training Method: LoRA with 8-bit quantization
• Dataset: MMLU humanities + synthetic data
• Specialization: History, philosophy, and humanities understanding

Available Quantizations

• FP16 (model-f16.gguf): Full precision, ~24GB
• Q8_0 (model-Q8_0.gguf): 8-bit quantization, ~13GB, high quality
• Q4_K_M (model-Q4_K_M.gguf): 4-bit quantization, ~7GB, recommended for most users

Usage with llama.cpp

# Download model
huggingface-cli download ambrosfitz/mistral-nemo-gguf model-Q4_K_M.gguf --local-dir ./models

# Run inference
./llama.cpp/main -m ./models/model-Q4_K_M.gguf -p "Question: What was the Renaissance?\n\nAnswer:" -n 256

Usage with Ollama

# Create Modelfile
cat > Modelfile <<EOF
FROM ./model-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9
EOF

# Create model
ollama create mistral-humanities -f Modelfile

# Run
ollama run mistral-humanities "What was the Renaissance?"

Example Prompts

Question: What were the main causes of World War I?

Answer:

Question: Explain the philosophical ideas of the Enlightenment.

Answer:

Question: Who wrote 'The Republic' and what is it about?

Answer:

Limitations

This model is specialized for humanities topics and may not perform as well on:

• Technical/scientific questions
• Mathematics
• Coding
• Current events (knowledge cutoff applies)

License

Apache 2.0 (same as base model)