Downloads · 30 days
837
8% of all-time downloads
Firefly-v4-BetterGGUF
Guilherme34/Firefly-v4-BetterGGUF
Firefly-v4-BetterGGUF is a machine learning model from Guilherme34. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as other.
All-time downloads
9.9K
Public
Repo size
49.1 GB
Likes
5
Public
Repo makeup
Click a slice to open those files.
.gguf49.1 GB · 100%
At a glance
- Library
- transformers
- License
- other
- Access
- Public
- Created
- Apr 24, 2026
- Updated
- Jul 1, 2026
- SHA
- 4ec603af
Base models
Model card
From the Hugging Face model README
<div align="center">
<div style="max-width:1040px;margin:0 auto;font-family:Inter,ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;">
<div style="position:relative;overflow:hidden;background:radial-gradient(circle at 50% 0%,#31577f 0%,#162033 42%,#080b12 100%);border:1px solid #334155;border-radius:30px;padding:58px 30px 46px;text-align:center;box-shadow:0 18px 80px rgba(15,23,42,.55);">
<svg width="104" height="104" viewBox="0 0 104 104" fill="none" style="margin-bottom:22px;">
<defs>
<linearGradient id="firefly_main_gradient" x1="18" y1="10" x2="86" y2="94" gradientUnits="userSpaceOnUse">
<stop stop-color="#60A5FA"/>
<stop offset=".45" stop-color="#A78BFA"/>
<stop offset="1" stop-color="#FBBF24"/>
</linearGradient>
<radialGradient id="firefly_core_gradient" cx="0" cy="0" r="1" gradientUnits="userSpaceOnUse" gradientTransform="translate(52 52) rotate(90) scale(38)">
<stop stop-color="#FDE68A"/>
<stop offset=".55" stop-color="#60A5FA"/>
<stop offset="1" stop-color="#172033" stop-opacity="0"/>
</radialGradient>
</defs>
<circle cx="52" cy="52" r="42" fill="url(#firefly_core_gradient)" opacity=".35">
<animate attributeName="opacity" values=".22;.48;.22" dur="3.8s" repeatCount="indefinite"/>
</circle>
<circle cx="52" cy="52" r="36" stroke="url(#firefly_main_gradient)" stroke-width="2" opacity=".48">
<animate attributeName="r" values="32;39;32" dur="4.5s" repeatCount="indefinite"/>
<animate attributeName="opacity" values=".26;.62;.26" dur="4.5s" repeatCount="indefinite"/>
</circle>
<path d="M52 18L61.6 40.2L85.7 43.1L67.8 59.2L72.5 83L52 70.7L31.5 83L36.2 59.2L18.3 43.1L42.4 40.2L52 18Z" stroke="url(#firefly_main_gradient)" stroke-width="3" stroke-linejoin="round">
<animate attributeName="opacity" values=".72;1;.72" dur="2.6s" repeatCount="indefinite"/>
</path>
</svg>
<h1 style="margin:0;color:#ffffff;font-size:60px;line-height:1;font-weight:950;letter-spacing:-2px;">
Firefly-v4 GGUF
</h1>
<p style="margin:20px auto 0;max-width:790px;color:#cbd5e1;font-size:19px;line-height:1.8;">
Polished <strong>GGUF quantizations</strong> of <strong>Firefly-v4</strong>, with experimental
<strong>Multi-Token Prediction</strong> support for compatible inference backends.
</p>
<div style="margin:32px auto 0;display:flex;flex-wrap:wrap;justify-content:center;align-items:center;gap:10px;max-width:900px;">
<span style="display:inline-flex;align-items:center;gap:8px;background:linear-gradient(180deg,#172033,#111827);border:1px solid #31435d;color:#93c5fd;padding:10px 17px;border-radius:999px;font-weight:800;box-shadow:inset 0 1px 0 rgba(255,255,255,.05);">
<svg width="17" height="17" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M4 4h16v16H4z"/>
<path d="M9 9h6v6H9z"/>
<animateTransform attributeName="transform" type="scale" values="1;1.08;1" dur="2.8s" repeatCount="indefinite"/>
</svg>
GGUF
</span>
<span style="display:inline-flex;align-items:center;gap:8px;background:linear-gradient(180deg,#1f1c12,#111827);border:1px solid #5b4620;color:#fbbf24;padding:10px 17px;border-radius:999px;font-weight:800;box-shadow:inset 0 1px 0 rgba(255,255,255,.05);">
<svg width="17" height="17" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M5 12h14"/>
<path d="M13 6l6 6-6 6"/>
<animateTransform attributeName="transform" type="translate" values="0 0;2 0;0 0" dur="1.8s" repeatCount="indefinite"/>
</svg>
MTP
</span>
<span style="display:inline-flex;align-items:center;gap:8px;background:linear-gradient(180deg,#10251a,#111827);border:1px solid #315f42;color:#86efac;padding:10px 17px;border-radius:999px;font-weight:800;box-shadow:inset 0 1px 0 rgba(255,255,255,.05);">
<svg width="17" height="17" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M2 12s4-6 10-6 10 6 10 6-4 6-10 6S2 12 2 12z"/>
<circle cx="12" cy="12" r="3">
<animate attributeName="r" values="2.25;3.35;2.25" dur="2.4s" repeatCount="indefinite"/>
</circle>
</svg>
Vision
</span>
<span style="display:inline-flex;align-items:center;gap:8px;background:linear-gradient(180deg,#1c1730,#111827);border:1px solid #4d3d72;color:#c4b5fd;padding:10px 17px;border-radius:999px;font-weight:800;box-shadow:inset 0 1px 0 rgba(255,255,255,.05);">
<svg width="17" height="17" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<polygon points="11 5 6 9 2 9 2 15 6 15 11 19"/>
<path d="M15.5 8.5a5 5 0 010 7">
<animate attributeName="opacity" values=".35;1;.35" dur="1.6s" repeatCount="indefinite"/>
</path>
<path d="M18.5 5.5a9 9 0 010 13">
<animate attributeName="opacity" values=".2;.85;.2" dur="1.6s" repeatCount="indefinite"/>
</path>
</svg>
Audio
</span>
<span style="display:inline-flex;align-items:center;gap:8px;background:linear-gradient(180deg,#102132,#111827);border:1px solid #315d7a;color:#7dd3fc;padding:10px 17px;border-radius:999px;font-weight:800;box-shadow:inset 0 1px 0 rgba(255,255,255,.05);">
<svg width="17" height="17" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M12 2l2.8 6.2L21 9l-4.5 4 1.2 6-5.7-3.2L6.3 19l1.2-6L3 9l6.2-.8L12 2z"/>
<animateTransform attributeName="transform" type="rotate" values="0 12 12;5 12 12;0 12 12;-5 12 12;0 12 12" dur="3.2s" repeatCount="indefinite"/>
</svg>
Gemma 4
</span>
<span style="display:inline-flex;align-items:center;gap:8px;background:linear-gradient(180deg,#2b1430,#111827);border:1px solid #69336e;color:#f0abfc;padding:10px 17px;border-radius:999px;font-weight:800;box-shadow:inset 0 1px 0 rgba(255,255,255,.05);">
<svg width="17" height="17" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M21 15a2 2 0 01-2 2H7l-4 4V5a2 2 0 012-2h14a2 2 0 012 2z"/>
<animate attributeName="opacity" values=".65;1;.65" dur="2.6s" repeatCount="indefinite"/>
</svg>
Roleplay
</span>
<span style="display:inline-flex;align-items:center;gap:8px;background:linear-gradient(180deg,#171a35,#111827);border:1px solid #3f4a8a;color:#a5b4fc;padding:10px 17px;border-radius:999px;font-weight:800;box-shadow:inset 0 1px 0 rgba(255,255,255,.05);">
<svg width="17" height="17" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M9 18h6"/>
<path d="M10 22h4"/>
<path d="M12 2a7 7 0 00-4 13c.8.8 1.5 1.7 1.8 3h4.4c.3-1.3 1-2.2 1.8-3A7 7 0 0012 2z"/>
<animate attributeName="opacity" values=".58;1;.58" dur="2s" repeatCount="indefinite"/>
</svg>
Reasoning
</span>
</div>
<div style="margin:34px auto 0;display:grid;grid-template-columns:repeat(auto-fit,minmax(180px,1fr));gap:12px;max-width:840px;">
<div style="background:rgba(15,23,42,.72);border:1px solid rgba(148,163,184,.22);border-radius:18px;padding:16px;">
<div style="color:#94a3b8;font-size:12px;font-weight:800;text-transform:uppercase;letter-spacing:.12em;">Format</div>
<div style="color:#ffffff;font-size:21px;font-weight:900;margin-top:4px;">GGUF</div>
</div>
<div style="background:rgba(15,23,42,.72);border:1px solid rgba(148,163,184,.22);border-radius:18px;padding:16px;">
<div style="color:#94a3b8;font-size:12px;font-weight:800;text-transform:uppercase;letter-spacing:.12em;">Feature</div>
<div style="color:#ffffff;font-size:21px;font-weight:900;margin-top:4px;">MTP Ready</div>
</div>
<div style="background:rgba(15,23,42,.72);border:1px solid rgba(148,163,184,.22);border-radius:18px;padding:16px;">
<div style="color:#94a3b8;font-size:12px;font-weight:800;text-transform:uppercase;letter-spacing:.12em;">Focus</div>
<div style="color:#ffffff;font-size:21px;font-weight:900;margin-top:4px;">Creative</div>
</div>
<div style="background:rgba(15,23,42,.72);border:1px solid rgba(148,163,184,.22);border-radius:18px;padding:16px;">
<div style="color:#94a3b8;font-size:12px;font-weight:800;text-transform:uppercase;letter-spacing:.12em;">Mode</div>
<div style="color:#ffffff;font-size:21px;font-weight:900;margin-top:4px;">Multimodal</div>
</div>
</div>
</div>
</div>
</div>
<div style="max-width:1040px;margin:0 auto;font-family:Inter,ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;"> <div style="background:linear-gradient(180deg,#101827,#0b101b);border:1px solid #334155;border-radius:24px;padding:30px;margin:26px auto;box-shadow:0 12px 48px rgba(2,6,23,.35);"> <h2 style="margin:0 0 16px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#60a5fa" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M4 19.5A2.5 2.5 0 016.5 17H20"/> <path d="M4 4.5A2.5 2.5 0 016.5 2H20v20H6.5A2.5 2.5 0 014 19.5z"/> <animate attributeName="opacity" values=".65;1;.65" dur="3s" repeatCount="indefinite"/> </svg> About </h2> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> This repository contains <strong>GGUF conversions</strong> of <strong>Firefly-v4</strong>, a multimodal fine-tune based on <strong>p-e-w/gemma-4-E2B-it-heretic-ara</strong>. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> These files are intended for local inference with GGUF-compatible software such as: </p> <div style="display:flex;flex-wrap:wrap;gap:10px;margin:18px 0 8px;">
<div style="display:grid;grid-template-columns:repeat(auto-fit,minmax(235px,1fr));gap:16px;margin:26px auto;"> <div style="background:linear-gradient(180deg,#141a27,#0b101b);border:1px solid #3c3320;border-radius:22px;padding:24px;box-shadow:0 10px 32px rgba(2,6,23,.25);"> <h3 style="margin:0 0 12px;color:#fbbf24;display:flex;align-items:center;gap:10px;"> <svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#fbbf24" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M5 12h14"/> <path d="M13 6l6 6-6 6"/> <animateTransform attributeName="transform" type="translate" values="0 0;2 0;0 0" dur="1.8s" repeatCount="indefinite"/> </svg> MTP Support </h3> <p style="margin:0;color:#cbd5e1;line-height:1.75;"> Includes experimental <strong>Multi-Token Prediction</strong> support. Performance and compatibility depend on your inference backend. </p> </div> <div style="background:linear-gradient(180deg,#111d18,#0b101b);border:1px solid #24452f;border-radius:22px;padding:24px;box-shadow:0 10px 32px rgba(2,6,23,.25);"> <h3 style="margin:0 0 12px;color:#86efac;display:flex;align-items:center;gap:10px;"> <svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#86efac" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M2 12s4-6 10-6 10 6 10 6-4 6-10 6S2 12 2 12z"/> <circle cx="12" cy="12" r="3"> <animate attributeName="r" values="2.2;3.4;2.2" dur="2.4s" repeatCount="indefinite"/> </circle> </svg> Multimodal </h3> <p style="margin:0;color:#cbd5e1;line-height:1.75;"> Retains Gemma 4's native multimodal capabilities, including image and audio understanding. </p> </div> <div style="background:linear-gradient(180deg,#211426,#0b101b);border:1px solid #51305a;border-radius:22px;padding:24px;box-shadow:0 10px 32px rgba(2,6,23,.25);"> <h3 style="margin:0 0 12px;color:#f0abfc;display:flex;align-items:center;gap:10px;"> <svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#f0abfc" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M21 15a2 2 0 01-2 2H7l-4 4V5a2 2 0 012-2h14a2 2 0 012 2z"/> <animate attributeName="opacity" values=".65;1;.65" dur="2.6s" repeatCount="indefinite"/> </svg> Creative Focus </h3> <p style="margin:0;color:#cbd5e1;line-height:1.75;"> Optimized for creative writing, roleplay, long conversations, and general instruction following. </p> </div> <div style="background:linear-gradient(180deg,#151735,#0b101b);border:1px solid #383f7a;border-radius:22px;padding:24px;box-shadow:0 10px 32px rgba(2,6,23,.25);"> <h3 style="margin:0 0 12px;color:#a5b4fc;display:flex;align-items:center;gap:10px;"> <svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#a5b4fc" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M9 18h6"/> <path d="M10 22h4"/> <path d="M12 2a7 7 0 00-4 13c.8.8 1.5 1.7 1.8 3h4.4c.3-1.3 1-2.2 1.8-3A7 7 0 0012 2z"/> <animate attributeName="opacity" values=".55;1;.55" dur="2s" repeatCount="indefinite"/> </svg> Optional Reasoning </h3> <p style="margin:0;color:#cbd5e1;line-height:1.75;"> Supports the <code><|think|></code> tag for switchable reasoning on compatible backends. </p> </div> </div>
<div style="background:radial-gradient(circle at top left,rgba(244,114,182,.18),transparent 36%),linear-gradient(135deg,#141b2b,#1e293b);border:1px solid #475569;border-radius:26px;padding:32px;margin:26px auto;box-shadow:0 16px 60px rgba(2,6,23,.38);"> <h2 style="margin:0 0 16px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#f472b6" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M20.8 4.6a5.5 5.5 0 00-7.8 0L12 5.6l-1-1a5.5 5.5 0 00-7.8 7.8l1 1L12 21l7.8-7.6 1-1a5.5 5.5 0 000-7.8z"/> <animate attributeName="opacity" values=".58;1;.58" dur="2.4s" repeatCount="indefinite"/> </svg> A Small Note </h2> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> This GGUF release includes <strong>experimental MTP support</strong>. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> I was not originally planning to spend much time on Firefly-v4's GGUF release, but many people have enjoyed using the model. Because of that, I decided to give this repository the attention it deserves and polish it properly. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> That said, <strong>I still personally consider Firefly-v3 to be my best model overall.</strong> Everyone has different preferences, though, and I am happy to continue supporting Firefly-v4 for those who enjoy it. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;margin-bottom:0;"> Thank you to everyone who has shared feedback and supported the project. </p> </div>
<div style="display:grid;grid-template-columns:repeat(auto-fit,minmax(300px,1fr));gap:18px;margin:26px auto;"> <div style="background:linear-gradient(180deg,#101827,#0b101b);border:1px solid #334155;border-radius:24px;padding:26px;"> <h2 style="margin:0 0 18px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#fbbf24" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M12 2l2.8 6.2L21 9l-4.5 4 1.2 6-5.7-3.2L6.3 19l1.2-6L3 9l6.2-.8L12 2z"/> <animateTransform attributeName="transform" type="rotate" values="0 12 12;5 12 12;0 12 12;-5 12 12;0 12 12" dur="3.2s" repeatCount="indefinite"/> </svg> Recommended </h2> <table> <thead> <tr> <th align="left">Quant</th> <th align="left">Recommendation</th> </tr> </thead> <tbody> <tr> <td><strong>Q4_K_M</strong></td> <td>Recommended for most users</td> </tr> <tr> <td><strong>Q4_K_S</strong></td> <td>Fast and efficient</td> </tr> <tr> <td><strong>Q6_K</strong></td> <td>Excellent quality</td> </tr> <tr> <td><strong>Q8_0</strong></td> <td>Highest GGUF quality</td> </tr> <tr> <td><strong>F16</strong></td> <td>Maximum precision</td> </tr> </tbody> </table> </div> <div style="background:linear-gradient(180deg,#101827,#0b101b);border:1px solid #334155;border-radius:24px;padding:26px;"> <h2 style="margin:0 0 18px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#60a5fa" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M4 4h16v16H4z"/> <path d="M9 9h6v6H9z"/> <animateTransform attributeName="transform" type="scale" values="1;1.08;1" dur="2.8s" repeatCount="indefinite"/> </svg> Available </h2> <table> <thead> <tr> <th align="left">Type</th> <th align="left">Notes</th> </tr> </thead> <tbody> <tr> <td><strong>Q2_K</strong></td> <td>Smallest size</td> </tr> <tr> <td><strong>Q3_K_S</strong></td> <td>Low-memory systems</td> </tr> <tr> <td><strong>Q3_K_M</strong></td> <td>Balanced</td> </tr> <tr> <td><strong>Q3_K_L</strong></td> <td>Improved Q3 quality</td> </tr> <tr> <td><strong>IQ4_XS</strong></td> <td>Excellent efficiency</td> </tr> <tr> <td><strong>Q4_K_S</strong></td> <td>Fast</td> </tr> <tr> <td><strong>Q4_K_M</strong></td> <td>Recommended</td> </tr> <tr> <td><strong>Q5_K_S</strong></td> <td>Higher quality</td> </tr> <tr> <td><strong>Q5_K_M</strong></td> <td>Higher quality</td> </tr> <tr> <td><strong>Q6_K</strong></td> <td>Excellent quality</td> </tr> <tr> <td><strong>Q8_0</strong></td> <td>Near-lossless</td> </tr> <tr> <td><strong>F16</strong></td> <td>Full precision</td> </tr> </tbody> </table> </div> </div>
<div style="background:linear-gradient(180deg,#101827,#0b101b);border-left:5px solid #60a5fa;border-radius:0 22px 22px 0;padding:28px;margin:26px auto;box-shadow:0 12px 48px rgba(2,6,23,.32);"> <h2 style="margin:0 0 16px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#60a5fa" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="8" r="5"/> <path d="M20 21a8 8 0 00-16 0"/> <animate attributeName="opacity" values=".62;1;.62" dur="3s" repeatCount="indefinite"/> </svg> Attribution </h2> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> <strong>Original Model:</strong> Firefly-v4<br> <strong>Author:</strong> Guilherme34 </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> <strong>Base Model:</strong> p-e-w/gemma-4-E2B-it-heretic-ara </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;margin-bottom:0;"> This repository contains only GGUF conversions. All credit for the model training, fine-tuning, datasets, and original work belongs to the original author. </p> </div>
<div style="background:linear-gradient(180deg,#101827,#0b101b);border:1px solid #334155;border-radius:24px;padding:28px;margin:26px auto;box-shadow:0 12px 48px rgba(2,6,23,.32);"> <h2 style="margin:0 0 16px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#c4b5fd" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M14 2H6a2 2 0 00-2 2v16a2 2 0 002 2h12a2 2 0 002-2V8z"/> <path d="M14 2v6h6"/> <path d="M16 13H8"/> <path d="M16 17H8"/> <path d="M10 9H8"/> <animate attributeName="opacity" values=".7;1;.7" dur="2.8s" repeatCount="indefinite"/> </svg> License </h2> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> This repository is distributed under the <strong>Firefly-v4 Attribution License</strong>. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> Commercial use, quantizations, merges, and derivative works are permitted under the terms described in the included <strong>LICENSE</strong> file. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;margin-bottom:0;"> Please preserve the required attribution when redistributing or building upon this model. </p> </div> <div style="background: #111118; border-left: 3px solid #ff4500; border-radius: 0 12px 12px 0; padding: 20px 24px; margin-bottom: 24px;"> <strong>⚠️ Disclaimer</strong> <p style="color: #8888a0; font-size: 14px; margin: 8px 0 0 0;">This model is <strong>uncensored</strong> and will generate content without built-in refusals. It is intended for <strong>creative fiction and roleplay between consenting adults.</strong> The creator is not responsible for how the model is used. Do not use it to produce content that is illegal in your jurisdiction.</p> </div>
<div align="center"> <div style="background:radial-gradient(circle at 50% 0%,rgba(96,165,250,.22),transparent 40%),linear-gradient(135deg,#080b12,#111827 62%,#172033);border:1px solid #334155;border-radius:28px;padding:36px 28px;margin:26px auto 0;text-align:center;box-shadow:0 18px 70px rgba(2,6,23,.44);"> <svg width="54" height="54" viewBox="0 0 54 54" fill="none" style="margin-bottom:14px;"> <circle cx="27" cy="27" r="20" stroke="#60a5fa" stroke-width="2" opacity=".35"> <animate attributeName="r" values="17;22;17" dur="3s" repeatCount="indefinite"/> <animate attributeName="opacity" values=".2;.55;.2" dur="3s" repeatCount="indefinite"/> </circle> <path d="M27 8L32 20L45 22L35.5 31L38 44L27 37L16 44L18.5 31L9 22L22 20L27 8Z" stroke="#fbbf24" stroke-width="2.3" stroke-linejoin="round"> <animate attributeName="opacity" values=".7;1;.7" dur="2.4s" repeatCount="indefinite"/> </path> </svg> <h3 style="color:#ffffff;margin:0;font-size:24px;"> Thank you for using Firefly-v4 GGUF </h3> <p style="color:#cbd5e1;line-height:1.8;max-width:720px;margin:14px auto 18px;font-size:16px;"> If you enjoy this release, consider checking out the original model and supporting future development. </p> <p style="color:#cbd5e1;line-height:1.8;margin:0;"> <strong>Original Model:</strong> <a href="https://huggingface.co/Guilherme34/Firefly-v4" style="color:#60a5fa;font-weight:800;"> https://huggingface.co/Guilherme34/Firefly-v4 </a> </p> </div> </div> </div>
<div style="max-width:1040px;margin:0 auto;font-family:Inter,ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;"> <div style="background:linear-gradient(180deg,#101827,#0b101b);border:1px solid #334155;border-radius:24px;padding:30px;margin:26px auto;box-shadow:0 12px 48px rgba(2,6,23,.35);"> <h2 style="margin:0 0 16px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#60a5fa" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M4 19.5A2.5 2.5 0 016.5 17H20"/> <path d="M4 4.5A2.5 2.5 0 016.5 2H20v20H6.5A2.5 2.5 0 014 19.5z"/> <animate attributeName="opacity" values=".65;1;.65" dur="3s" repeatCount="indefinite"/> </svg> About </h2> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> This repository contains <strong>GGUF conversions</strong> of <strong>Firefly-v4</strong>, a multimodal fine-tune based on <strong>p-e-w/gemma-4-E2B-it-heretic-ara</strong>. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> These files are intended for local inference with GGUF-compatible software such as: </p> <div style="display:flex;flex-wrap:wrap;gap:10px;margin:18px 0 8px;">
<span style="background:#172033;border:1px solid #263244;color:#dbeafe;padding:8px 13px;border-radius:12px;font-weight:700;">llama.cpp</span> <span style="background:#172033;border:1px solid #263244;color:#dbeafe;padding:8px 13px;border-radius:12px;font-weight:700;">LM Studio</span> <span style="background:#172033;border:1px solid #263244;color:#dbeafe;padding:8px 13px;border-radius:12px;font-weight:700;">KoboldCpp</span> <span style="background:#172033;border:1px solid #263244;color:#dbeafe;padding:8px 13px;border-radius:12px;font-weight:700;">Jan</span> <span style="background:#172033;border:1px solid #263244;color:#dbeafe;padding:8px 13px;border-radius:12px;font-weight:700;">Text Generation WebUI</span> <span style="background:#172033;border:1px solid #263244;color:#dbeafe;padding:8px 13px;border-radius:12px;font-weight:700;">Ollama</span>
</div> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> Original model: <a href="https://huggingface.co/Guilherme34/Firefly-v4" style="color:#60a5fa;font-weight:800;"> https://huggingface.co/Guilherme34/Firefly-v4 </a> </p> </div><div style="display:grid;grid-template-columns:repeat(auto-fit,minmax(235px,1fr));gap:16px;margin:26px auto;"> <div style="background:linear-gradient(180deg,#141a27,#0b101b);border:1px solid #3c3320;border-radius:22px;padding:24px;box-shadow:0 10px 32px rgba(2,6,23,.25);"> <h3 style="margin:0 0 12px;color:#fbbf24;display:flex;align-items:center;gap:10px;"> <svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#fbbf24" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M5 12h14"/> <path d="M13 6l6 6-6 6"/> <animateTransform attributeName="transform" type="translate" values="0 0;2 0;0 0" dur="1.8s" repeatCount="indefinite"/> </svg> MTP Support </h3> <p style="margin:0;color:#cbd5e1;line-height:1.75;"> Includes experimental <strong>Multi-Token Prediction</strong> support. Performance and compatibility depend on your inference backend. </p> </div> <div style="background:linear-gradient(180deg,#111d18,#0b101b);border:1px solid #24452f;border-radius:22px;padding:24px;box-shadow:0 10px 32px rgba(2,6,23,.25);"> <h3 style="margin:0 0 12px;color:#86efac;display:flex;align-items:center;gap:10px;"> <svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#86efac" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M2 12s4-6 10-6 10 6 10 6-4 6-10 6S2 12 2 12z"/> <circle cx="12" cy="12" r="3"> <animate attributeName="r" values="2.2;3.4;2.2" dur="2.4s" repeatCount="indefinite"/> </circle> </svg> Multimodal </h3> <p style="margin:0;color:#cbd5e1;line-height:1.75;"> Retains Gemma 4's native multimodal capabilities, including image and audio understanding. </p> </div> <div style="background:linear-gradient(180deg,#211426,#0b101b);border:1px solid #51305a;border-radius:22px;padding:24px;box-shadow:0 10px 32px rgba(2,6,23,.25);"> <h3 style="margin:0 0 12px;color:#f0abfc;display:flex;align-items:center;gap:10px;"> <svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#f0abfc" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M21 15a2 2 0 01-2 2H7l-4 4V5a2 2 0 012-2h14a2 2 0 012 2z"/> <animate attributeName="opacity" values=".65;1;.65" dur="2.6s" repeatCount="indefinite"/> </svg> Creative Focus </h3> <p style="margin:0;color:#cbd5e1;line-height:1.75;"> Optimized for creative writing, roleplay, long conversations, and general instruction following. </p> </div> <div style="background:linear-gradient(180deg,#151735,#0b101b);border:1px solid #383f7a;border-radius:22px;padding:24px;box-shadow:0 10px 32px rgba(2,6,23,.25);"> <h3 style="margin:0 0 12px;color:#a5b4fc;display:flex;align-items:center;gap:10px;"> <svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#a5b4fc" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M9 18h6"/> <path d="M10 22h4"/> <path d="M12 2a7 7 0 00-4 13c.8.8 1.5 1.7 1.8 3h4.4c.3-1.3 1-2.2 1.8-3A7 7 0 0012 2z"/> <animate attributeName="opacity" values=".55;1;.55" dur="2s" repeatCount="indefinite"/> </svg> Optional Reasoning </h3> <p style="margin:0;color:#cbd5e1;line-height:1.75;"> Supports the <code><|think|></code> tag for switchable reasoning on compatible backends. </p> </div> </div>
<div style="background:radial-gradient(circle at top left,rgba(244,114,182,.18),transparent 36%),linear-gradient(135deg,#141b2b,#1e293b);border:1px solid #475569;border-radius:26px;padding:32px;margin:26px auto;box-shadow:0 16px 60px rgba(2,6,23,.38);"> <h2 style="margin:0 0 16px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#f472b6" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M20.8 4.6a5.5 5.5 0 00-7.8 0L12 5.6l-1-1a5.5 5.5 0 00-7.8 7.8l1 1L12 21l7.8-7.6 1-1a5.5 5.5 0 000-7.8z"/> <animate attributeName="opacity" values=".58;1;.58" dur="2.4s" repeatCount="indefinite"/> </svg> A Small Note </h2> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> This GGUF release includes <strong>experimental MTP support</strong>. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> I was not originally planning to spend much time on Firefly-v4's GGUF release, but many people have enjoyed using the model. Because of that, I decided to give this repository the attention it deserves and polish it properly. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> That said, <strong>I still personally consider Firefly-v3 to be my best model overall.</strong> Everyone has different preferences, though, and I am happy to continue supporting Firefly-v4 for those who enjoy it. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;margin-bottom:0;"> Thank you to everyone who has shared feedback and supported the project. </p> </div>
<div style="display:grid;grid-template-columns:repeat(auto-fit,minmax(300px,1fr));gap:18px;margin:26px auto;"> <div style="background:linear-gradient(180deg,#101827,#0b101b);border:1px solid #334155;border-radius:24px;padding:26px;"> <h2 style="margin:0 0 18px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#fbbf24" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M12 2l2.8 6.2L21 9l-4.5 4 1.2 6-5.7-3.2L6.3 19l1.2-6L3 9l6.2-.8L12 2z"/> <animateTransform attributeName="transform" type="rotate" values="0 12 12;5 12 12;0 12 12;-5 12 12;0 12 12" dur="3.2s" repeatCount="indefinite"/> </svg> Recommended </h2> <table> <thead> <tr> <th align="left">Quant</th> <th align="left">Recommendation</th> </tr> </thead> <tbody> <tr> <td><strong>Q4_K_M</strong></td> <td>Recommended for most users</td> </tr> <tr> <td><strong>Q4_K_S</strong></td> <td>Fast and efficient</td> </tr> <tr> <td><strong>Q6_K</strong></td> <td>Excellent quality</td> </tr> <tr> <td><strong>Q8_0</strong></td> <td>Highest GGUF quality</td> </tr> <tr> <td><strong>F16</strong></td> <td>Maximum precision</td> </tr> </tbody> </table> </div> <div style="background:linear-gradient(180deg,#101827,#0b101b);border:1px solid #334155;border-radius:24px;padding:26px;"> <h2 style="margin:0 0 18px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#60a5fa" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M4 4h16v16H4z"/> <path d="M9 9h6v6H9z"/> <animateTransform attributeName="transform" type="scale" values="1;1.08;1" dur="2.8s" repeatCount="indefinite"/> </svg> Available </h2> <table> <thead> <tr> <th align="left">Type</th> <th align="left">Notes</th> </tr> </thead> <tbody> <tr> <td><strong>Q2_K</strong></td> <td>Smallest size</td> </tr> <tr> <td><strong>Q3_K_S</strong></td> <td>Low-memory systems</td> </tr> <tr> <td><strong>Q3_K_M</strong></td> <td>Balanced</td> </tr> <tr> <td><strong>Q3_K_L</strong></td> <td>Improved Q3 quality</td> </tr> <tr> <td><strong>IQ4_XS</strong></td> <td>Excellent efficiency</td> </tr> <tr> <td><strong>Q4_K_S</strong></td> <td>Fast</td> </tr> <tr> <td><strong>Q4_K_M</strong></td> <td>Recommended</td> </tr> <tr> <td><strong>Q5_K_S</strong></td> <td>Higher quality</td> </tr> <tr> <td><strong>Q5_K_M</strong></td> <td>Higher quality</td> </tr> <tr> <td><strong>Q6_K</strong></td> <td>Excellent quality</td> </tr> <tr> <td><strong>Q8_0</strong></td> <td>Near-lossless</td> </tr> <tr> <td><strong>F16</strong></td> <td>Full precision</td> </tr> </tbody> </table> </div> </div>
<div style="background:linear-gradient(180deg,#101827,#0b101b);border-left:5px solid #60a5fa;border-radius:0 22px 22px 0;padding:28px;margin:26px auto;box-shadow:0 12px 48px rgba(2,6,23,.32);"> <h2 style="margin:0 0 16px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#60a5fa" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <circle cx="12" cy="8" r="5"/> <path d="M20 21a8 8 0 00-16 0"/> <animate attributeName="opacity" values=".62;1;.62" dur="3s" repeatCount="indefinite"/> </svg> Attribution </h2> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> <strong>Original Model:</strong> Firefly-v4<br> <strong>Author:</strong> Guilherme34 </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> <strong>Base Model:</strong> p-e-w/gemma-4-E2B-it-heretic-ara </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;margin-bottom:0;"> This repository contains only GGUF conversions. All credit for the model training, fine-tuning, datasets, and original work belongs to the original author. </p> </div>
<div style="background:linear-gradient(180deg,#101827,#0b101b);border:1px solid #334155;border-radius:24px;padding:28px;margin:26px auto;box-shadow:0 12px 48px rgba(2,6,23,.32);"> <h2 style="margin:0 0 16px;color:#ffffff;display:flex;align-items:center;gap:12px;"> <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="#c4b5fd" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"> <path d="M14 2H6a2 2 0 00-2 2v16a2 2 0 002 2h12a2 2 0 002-2V8z"/> <path d="M14 2v6h6"/> <path d="M16 13H8"/> <path d="M16 17H8"/> <path d="M10 9H8"/> <animate attributeName="opacity" values=".7;1;.7" dur="2.8s" repeatCount="indefinite"/> </svg> License </h2> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> This repository is distributed under the <strong>Firefly-v4 Attribution License</strong>. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;"> Commercial use, quantizations, merges, and derivative works are permitted under the terms described in the included <strong>LICENSE</strong> file. </p> <p style="color:#cbd5e1;line-height:1.8;font-size:16px;margin-bottom:0;"> Please preserve the required attribution when redistributing or building upon this model. </p> </div> <div style="background: #111118; border-left: 3px solid #ff4500; border-radius: 0 12px 12px 0; padding: 20px 24px; margin-bottom: 24px;"> <strong>⚠️ Disclaimer</strong> <p style="color: #8888a0; font-size: 14px; margin: 8px 0 0 0;">This model is <strong>uncensored</strong> and will generate content without built-in refusals. It is intended for <strong>creative fiction and roleplay between consenting adults.</strong> The creator is not responsible for how the model is used. Do not use it to produce content that is illegal in your jurisdiction.</p> </div>
<div align="center"> <div style="background:radial-gradient(circle at 50% 0%,rgba(96,165,250,.22),transparent 40%),linear-gradient(135deg,#080b12,#111827 62%,#172033);border:1px solid #334155;border-radius:28px;padding:36px 28px;margin:26px auto 0;text-align:center;box-shadow:0 18px 70px rgba(2,6,23,.44);"> <svg width="54" height="54" viewBox="0 0 54 54" fill="none" style="margin-bottom:14px;"> <circle cx="27" cy="27" r="20" stroke="#60a5fa" stroke-width="2" opacity=".35"> <animate attributeName="r" values="17;22;17" dur="3s" repeatCount="indefinite"/> <animate attributeName="opacity" values=".2;.55;.2" dur="3s" repeatCount="indefinite"/> </circle> <path d="M27 8L32 20L45 22L35.5 31L38 44L27 37L16 44L18.5 31L9 22L22 20L27 8Z" stroke="#fbbf24" stroke-width="2.3" stroke-linejoin="round"> <animate attributeName="opacity" values=".7;1;.7" dur="2.4s" repeatCount="indefinite"/> </path> </svg> <h3 style="color:#ffffff;margin:0;font-size:24px;"> Thank you for using Firefly-v4 GGUF </h3> <p style="color:#cbd5e1;line-height:1.8;max-width:720px;margin:14px auto 18px;font-size:16px;"> If you enjoy this release, consider checking out the original model and supporting future development. </p> <p style="color:#cbd5e1;line-height:1.8;margin:0;"> <strong>Original Model:</strong> <a href="https://huggingface.co/Guilherme34/Firefly-v4" style="color:#60a5fa;font-weight:800;"> https://huggingface.co/Guilherme34/Firefly-v4 </a> </p> </div> </div> </div>
- Library
- transformers
- License
- other
- Languages
- en
- Created
- Apr 24, 2026
- Updated
- Jul 1, 2026