morpheus-gguf

Morpheus (Mamba-2) — Basque FIM Autocomplete (GGUF)

GGUF quantized version of Morpheus v3, a 91M-parameter Mamba-2 language model for Basque (Euskara) text autocompletion with Fill-in-the-Middle (FIM) support — the model can complete text at the cursor, not just at the end.

This is a continued-pretraining of the AR-only v2 model (step 74K) on FIM-structured data.

What's new in v3 (FIM)

FIM training recipe (Phase 6 v3)

• Base: v2 best.pt (AR checkpoint, step 3500 of Phase 6 v2)
• Data: 500M tokens, 70/30 FIM/AR ratio (BigCode-style token-level splitting)
• EOT loss weighting: 5× on the <EOT> stop token — forces the model to learn when to stop generating infill, not just what to generate
• Packed dataset: Greedy packing of whole examples into 1025-token windows (no cross-example contamination)
• LR: 1.0e-3, cosine decay, ~3,815 steps
• Tokenizer: basque_unigram_fim.model (original 4000 pieces + 4 FIM tokens)

The 70/30 ratio + 5× EOT weight was selected after a senior ML engineer review of the v2 failure mode (unreliable <EOT> emission → over-generation). See Option D in the analysis below.

Available Files

FIM Evaluation Results

Trained-for-behavior eval on 147 held-out FIM examples (token-level splits, 20% at linguistic boundaries):

Key findings:

• The 5× EOT loss weighting worked as designed: EOT reliability up 77% → 88%, and the model stopped over-generating (56.8 → 40.3 chars vs 45.0 target).
• The feared failure mode — premature truncation from over-weighting EOT — did not materialize (1.4% overall, 2.3% in the long bucket; failure threshold was 15%).
• AR perplexity is stable (7.4 → 7.5) — "FIM-for-free" property holds even with 70% FIM data.

Model Details

• Architecture: Mamba-2 (State Space Model), 24 layers, d_model 768
• Parameters: 91M
• Checkpoint: Phase 6 v3 best.pt (step 3500)
• AR perplexity: 7.5 | FIM perplexity: 7.9
• Tokenizer: basque_unigram_fim.model — 4004 pieces (original 4000 + <PRE> <SUF> <MID> <EOT>)
• Trained without BOS token (add_bos_token=false)

FIM Token Format

The model uses Code Llama-style FIM tokens. To do a fill-in-the-middle completion, structure the prompt as:

<PRE>{prefix}<SUF>{suffix}<MID>

The model generates the infill and emits <EOT> when done.

Usage

With the Morpheus demo server (recommended)

The Morpheus demo includes a FastAPI proxy that handles FIM templating, token-ID encoding, and an OpenAI-compatible API:

cd demo
docker compose -f docker-compose.yml -f docker-compose.local.yml up -d --build
# Open http://localhost:9090/editor.html

Set MORPHEUS_MODEL=v3_fim.Q5_K_M.gguf to use this model.

Direct llama-server (raw FIM)

llama-server -m v3_fim.Q5_K_M.gguf --host 127.0.0.1 --port 8080 -ngl 0 --threads 8

# FIM completion:
curl http://localhost:8080/completion \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "<PRE>Kaixo, <SUF> moduz?<MID>",
    "n_predict": 20,
    "temperature": 0.2,
    "top_k": 5,
    "stop": ["<EOT>"]
  }'

⚠️ Critical: token-ID prompts, not strings

As with v2, deploy via token-ID prompts (encode with sentencepiece using basque_unigram_fim.model, send IDs to /completion). The proxy handles this automatically. See the v2 model card for the full explanation.

Decoding parameters (recommended)

These are wired into the demo proxy and editor:

Intended Use

Desktop text-editor ghost-text autocompletion for Basque prose. The Mamba-2 architecture's O(1) decode cost makes it well-suited to long editing sessions where latency per token matters more than parallelism.

Not intended for: instruction following, chat, translation, or factual QA. This is a narrow autocomplete model.

License

Apache-2.0.

Citation

Morpheus v3 — Mamba-2 FIM Autocomplete for Basque.
itzune/morpheus-gguf, 2026.