LFM2-8B-A1B-GLM-4.7-Flash-Thinking-Quantum-IQ1C

<small><B style="font-family: 'Arial Black', sans-serif; font-weight: 900; color: #f1c40f; text-decoration: none; text-shadow: 1px 1px 0 #d35400, 2px 2px 0 #d35400, 3px 3px 0 #d35400, 4px 4px 0 #d35400, 5px 5px 0 #d35400, 6px 6px 7px rgba(0,0,0,0.4);">[DavidAU]</B> <a href="https://huggingface.co/DavidAU" target="_blank" style="padding-left:5px">FOLLOW</a> | <a href="https://huggingface.co/DavidAU" target="_blank">ALL MODELS</a> | <a href="https://huggingface.co/DavidAU/collections" target="_blank">COLLECTIONS</a> | <a href="https://huggingface.co/models?sort=trending&search=DavidAU" target="_blank">TRENDING</a> | <a href="https://huggingface.co/DavidAU/support" target="_blank">FAQs</a> | <a href="https://huggingface.co/DavidAU/support" target="_blank">SUPPORT</a> | <a href="https://buymeacoffee.com/davidbelton" target="_blank">Buy Me A Coffee (😁)</a></small>

<h2>LFM2-8B-A1B-GLM-4.7-Flash-Thinking-Quantum-IQ1C/</h2>

Fine tune of "LFM2-8B-A1B" using Unsloth using custom dataset(s), 128k context in 16 bit precision.

This model is a sparse mixture of experts model (32) with 4 experts activated.

Speed exceeds 50-100 t/s on CPU // 200 t/s on most cards // 400 t/s + on 5090 at QUANT Q6K [4 experts].

One example generation below.

Can also be used on phones // mobile devices.

IN HOUSE BENCHMARKS [by Nightmedia]:

         arc-c arc/e boolq hswag obkqa piqa  wino

LFM2-8B-A1B-GLM-4.7-Flash-Thinking-Quantum-IQ1C
mxfp8    0.495,0.709,0.759,0.658,0.404,0.764,0.596

---

BASE UNTUNED MODEL:

LFM2-8B-A1B
mxfp8    0.460,0.575,0.829,0.624,0.394,0.711,0.567

---

<H2>EXAMPLE GENERATION: [4 experts, Q6K]</H2>