Nexus-TinyFunction-1.2B-v2.0-Haswell-untrained-GGUF

Nexus-TinyFunction-1.2B-v2.0 — untrained Haswell GGUF exports

This is not a trained/QAT model and not a 50 tok/s claim. These are untrained GGUF requantizations from the original BF16 checkpoint nexus-syntegra/Nexus-TinyFunction-1.2B-v2.0 revision baba0231ae34cc42f9eccb4c657cdea54c267efe.

Converted with llama.cpp 4d9176092d00586775af140581bb0b558ddc4389 (convert_hf_to_gguf.py → F16 → llama-quantize). Not requantized from the publisher Q4 GGUF.

Haswell VPS timing has not been run. Use these files only after a reviewed models.lock.json update. The appliance currently allowlists publisher GGUFs only.

Priority file for VPS timing

Tensor types: 92×Q4_0, 55×F32, 1×Q6_K (default embed, not --pure).

Separate --pure Q4_0 companion (do not replace the file above)

Requested by the CPU Phase 1 spec. Own quality and VPS speed gate. token_embd.weight is Q4_0 (tied output included). No Q6_K. Norms and short-conv kernels stay F32. Smaller file size is not a speed claim.

Tensor types: 93×Q4_0, 55×F32.

Other untrained candidates in this repo

Our Q4_K_M is a reconstruction from our F16 export. It is not the publisher file (693d8cb743ef1857ea2461bee211d98a63cd9b73fa03e8ad2484d63236f3f32e).

Identity

• Architecture: Lfm2ForCausalLM, 16 hybrid blocks (10 short-conv, 6 attn)
• Width 2048, 32 heads, 8 KV heads, vocab 65536, tied embeddings
• Config intermediate_size 12288 with auto-adjust; actual FFN is 8192
• Card context 32768; config max_position_embeddings 128000 is not proof of trained 128K
• Prompt protocol: ChatML + XML <tools> / <tool_call> from cmd/nexus/prompt.go
• Do not use the base tokenizer tools= chat template

Phase 1 quality screen (24 newly authored examples, not BFCL)

Q4_0 lost two single-call argument cases vs BF16. That is a material quality drop on a small screen. QAT is justified only if this Q4_0 is faster on the Haswell VPS.

A local Mac llama-cli screen of the --pure companion scored 20/24 (83.3%). That is not an H100 or VPS number and is not a quality recovery. The VPS gateway must re-score --pure before any lock change.

License

Fine-tune card: Apache-2.0. Base weights: LiquidAI LFM2.5-1.2B-Instruct under LFM Open License v1.0 (LICENSE-LFM). Review both before further distribution.

Suggested lock snippet (not applied)

"Q4_0_UNTRAINED": {
  "file": "nexus-tinyfunction-1.2b-v2.0-untrained-q4_0.gguf",
  "bytes": 695751584,
  "sha256": "a00bb3d4ad3650d12f54bdd66b12ad5c9d5b9f3b338a8422a221715edeaa7006"
}

Repository for a lock update would be this repo id and the uploaded revision SHA. Do not deploy until the CPU owner reviews the lock/loader change.