emma-5-GGUF

Emma-5 GGUF

Unofficial GGUF conversion of eldavoo/emma-5, archived from the original Egomnia Emma-5 release.

The conversion was validated by reconstructing the ONNX attention graph in PyTorch and comparing logits against the original ONNX Runtime output before exporting to GGUF.

The original ONNX model remains the reference runtime. This GGUF is intended for raw llama.cpp completion use, not chat-template frontends.

Files

Conversion Details

The original model is distributed as ONNX only. Direct conversion to GGUF required matching the exported attention graph exactly.

Pipeline summary:

1. ONNX weights and rotary constants were loaded.
2. The exported attention graph was matched in PyTorch for validation.
3. Logit parity was checked against ONNX Runtime.
4. Q/K rows were permuted from split-half RoPE layout to llama.cpp interleaved layout.
5. The model was exported as FP16 GGUF with rms_norm_eps=1e-6.

Validation result:

max absolute logit error: ~4e-5

Architecture

Prompt Format

Use a single-line Italian instruction format:

### Istruzione: Qual e la capitale d Italia? ### Risposta:

Use raw completion mode only.

Do not use ChatML, Llama 3 chat templates, OpenAI-style messages, or frontend "conversation" wrappers. In particular, prompts containing markers like <|im_start|> or <|im_end|> are not supported and may cause repeated marker output.

Avoid multiline templates in llama.cpp. The original ONNX SentencePiece path normalizes whitespace differently, while llama.cpp may preserve newline tokens.

For the most faithful behavior, compare against or use the original ONNX Runtime model.

Example

./llama-cli \
  -m emma-5-f16.gguf \
  --no-conversation \
  -p "### Istruzione: Qual e la capitale d Italia? ### Risposta:" \
  -n 64 \
  --temp 0.7 \
  --repeat-penalty 1.2

Expected style:

La capitale d'Italia è Roma...

Quantization

This repo provides the FP16 GGUF. Other quantizations can be generated with llama.cpp:

./llama-quantize emma-5-f16.gguf emma-5-Q4_K_M.gguf Q4_K_M

Compatibility

Converted and smoke-tested with a recent llama.cpp build.

Credits

• Original model by Egomnia S.p.A.
• Archived by eldavoo
• GGUF conversion by Spadav

This is an unofficial conversion.