Downloads · 30 days
451
56% of all-time downloads
Sparset/Synth-v1-qwen3.5-4b-GGUF
Synth-v1-qwen3.5-4b-GGUF is a image-text-to-text model from Sparset. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as other.
<img src="synth-gguf-banner.png" alt="Synth v1 GGUF: Q4KM and Q80 for llama.cpp" style="width:100%;max-width:100%;height:auto;display:block;margin-bottom:1em;" /
Downloads · 30 days
451
56% of all-time downloads
All-time downloads
811
Public
Repo size
7.9 GB
Likes
2
Public
Click a slice to open those files.
.gguf7.9 GB · 100%
From the Hugging Face model README
[!NOTE] GGUF quantizations of Sparset/Synth-v1-qwen3.5-4b, a document-synthesis task model fine-tuned from Qwen3.5-4B. Synth is trained to answer from documents you supply in the prompt: it combines evidence, resolves conflicting or superseded records, does grounded calculations, and flags insufficient evidence. These files are for llama.cpp and compatible local runtimes.
Research release — non-commercial use only. Same terms as the source model. See License.
These quantizations were not benchmarked. The results below were measured on the source BF16 checkpoint. The GGUF files passed load-and-generate smoke tests only. Quantization may change quality, so evaluate the file you deploy.
<video src="https://huggingface.co/Sparset/Synth-v1-qwen3.5-4b-GGUF/resolve/main/synth-side-by-side.mp4" controls autoplay muted loop playsinline style="width:100%;height:auto;display:block;"></video>
Recorded with the BF16 source model, not these GGUF files. Three cases from the internal 500-case benchmark, with the same question, the same six documents, greedy decoding and thinking off for both models: a cross-document calculation (Synth correct, the base model not), an insufficient-evidence case (both decline; only Synth leaves out unsupported citations) and a cross-document eligibility case (Synth correct here, but this task is mixed and remains its weakest). The source model card explains each case and how typical it is.
Measured on the BF16 source model, not these GGUF files. Synth and its base model, Qwen3.5-4B, answered the same 500 internal synthetic cases with identical prompts, greedy decoding and thinking off. Every document for a case was supplied in the prompt, so retrieval was not tested. Synth improved on 7 of 10 task types and in all 10 enterprise domains.
<style> .tl-table { display:table !important; width:100% !important; table-layout:fixed !important; border-collapse:separate; border-spacing:0; overflow:hidden; border:1px solid rgba(246,246,243,0.16); border-radius:0; background:#090909; font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,sans-serif; font-size:13px; color:rgba(246,246,243,0.80); } .tl-table th { padding:10px 6px; text-align:center; font-weight:600; color:rgba(246,246,243,0.55); background:#101010; border-right:1px solid rgba(246,246,243,0.16); border-bottom:1px solid rgba(246,246,243,0.16); } .tl-table td { padding:10px 6px; text-align:center; color:rgba(246,246,243,0.80); background:#090909; border-right:1px solid rgba(246,246,243,0.16); border-bottom:1px solid rgba(246,246,243,0.16); vertical-align:middle; font-variant-numeric:tabular-nums; } .tl-table tr > :last-child { border-right:0; } .tl-table tbody tr:last-child td { border-bottom:0; } .tl-table .group { color:#F6F6F3; font-weight:700; border-bottom:2px solid #3E61EB; } .tl-table .name { padding-left:16px; text-align:left; color:#F6F6F3; font-weight:600; } .tl-table .tl { background:rgba(62,97,235,0.16); } .tl-table th.tl { color:#8DA2F5; } .tl-table td.tl strong, .tl-table td.tl { color:#F6F6F3; } .tl-table strong { color:#F6F6F3; } .tl-table .up { color:#8DA2F5; font-weight:700; } .tl-table .section { padding:10px 16px; text-align:left; color:#8DA2F5; background:#101010; font-weight:700; letter-spacing:.06em; text-transform:uppercase; font-size:12px; } .tl-table .avg td { border-top:2px solid #3E61EB; font-weight:700; } @media (max-width: 640px) { .tl-table { display:block !important; overflow-x:auto !important; table-layout:auto !important; } .tl-table th, .tl-table td { min-width:92px; } .tl-table th:first-child, .tl-table td:first-child { min-width:140px; } } .model-card-content blockquote.note, blockquote.note { border-width:0 !important; box-shadow:inset 4px 0 0 #3E61EB !important; background:rgba(62,97,235,0.14) !important; border-radius:0 !important; padding:12px 18px 12px 22px !important; } blockquote.note p { margin:0 !important; font-style:normal !important; } </style> <table class="tl-table"> <thead> <tr><th style="width:30%;text-align:left;padding-left:16px;">Metric</th><th class="tl">Synth v1 (BF16)</th><th>Qwen3.5-4B</th><th>Change</th></tr> </thead> <tbody> <tr><td class="name">Composite score (0–100)</td><td class="tl"><strong>95.22</strong></td><td>91.16</td><td>+4.06</td></tr> <tr><td class="name">Content-check pass rate</td><td class="tl"><strong>95.8%</strong></td><td>95.0%</td><td>+0.8 pp</td></tr> <tr><td class="name">Citation F1 (0–100)</td><td class="tl"><strong>90.27</strong></td><td>83.45</td><td>+6.82</td></tr> <tr><td class="name">Response-status correctness</td><td class="tl"><strong>100.0%</strong></td><td>89.4%</td><td>+10.6 pp</td></tr> <tr><td class="name">JSON compliance</td><td class="tl"><strong>95.4%</strong></td><td>91.4%</td><td>+4.0 pp</td></tr> </tbody> </table> <table class="tl-table"> <thead> <tr><th style="width:30%;text-align:left;padding-left:16px;">Task (50 cases each)</th><th class="tl">Synth v1 (BF16)</th><th>Qwen3.5-4B</th><th>Change</th></tr> </thead> <tbody> <tr><td class="name">Insufficient evidence</td><td class="tl"><strong>94.60</strong></td><td>75.50</td><td>+19.10</td></tr> <tr><td class="name">Cross-document calculation</td><td class="tl"><strong>98.70</strong></td><td>85.30</td><td>+13.40</td></tr> <tr><td class="name">Authoritative-source conflict</td><td class="tl"><strong>99.83</strong></td><td>92.42</td><td>+7.42</td></tr> <tr><td class="name">Cross-document eligibility</td><td class="tl"><strong>71.17</strong></td><td>67.53</td><td>+3.63</td></tr> <tr><td class="name">Single-document fact</td><td class="tl"><strong>100.00</strong></td><td>99.53</td><td>+0.47</td></tr> <tr><td class="name">Prompt-injection resistance</td><td class="tl"><strong>100.00</strong></td><td>99.83</td><td>+0.17</td></tr> <tr><td class="name">Single-document owner</td><td class="tl"><strong>100.00</strong></td><td>99.83</td><td>+0.17</td></tr> <tr><td class="name">Clarification required</td><td class="tl">100.00</td><td>100.00</td><td>0.00</td></tr> <tr><td class="name">Conversational reference resolution</td><td class="tl">99.53</td><td><strong>100.00</strong></td><td>−0.47</td></tr> <tr><td class="name">Latest-version precedence</td><td class="tl">88.33</td><td><strong>91.67</strong></td><td>−3.33</td></tr> </tbody> </table>Not evaluated: these quantizations, retrieval-in-the-loop or agent use, inputs longer than 4,096 tokens, latency, and general chat, reasoning or safety benchmarks. The source model card has the per-domain results, full method and complete caveats.
| File | Quantization | Size | Notes |
|---|---|---|---|
Synth-v1-qwen3.5-4b-Q4_K_M.gguf | Q4_K_M | 2.71 GB | Smaller; suited to CPUs and low-memory GPUs |
Synth-v1-qwen3.5-4b-Q8_0.gguf | Q8_0 | 4.48 GB | Closer to the source weights; larger |
mmproj-Synth-v1-qwen3.5-4b-F16.gguf | F16 | 672 MB | Optional vision projector, unchanged from the base model; not needed for text use |
conversion_manifest.json | — | — | Source revision, converter revision, SHA-256 checksums, validation record |
| File | Bytes | SHA-256 |
|---|---|---|
Synth-v1-qwen3.5-4b-Q4_K_M.gguf | 2,708,803,840 | 3c7958590600d0a1b399fcf035e8feeefbcd382d7eb20043fefcd53d8fed7e90 |
Synth-v1-qwen3.5-4b-Q8_0.gguf | 4,482,402,560 | 3536782b2aab502255b45d088d848fefe5e62b2a03a36ee754350cb653c2f770 |
mmproj-Synth-v1-qwen3.5-4b-F16.gguf | 672,422,944 | a4e436d3b61e0b8344b41506962636d541ed5c587f5709d0c93d533b81e93de5 |
Sparset/Synth-v1-qwen3.5-4b at revision
3d719e0f6b0e8364624e905d4068c77e8148e52d, the same revision that was benchmarked.6a1a922d269908a29cbd4b49c27e6a8e7fd10fae.--no-mtp, so the
multi-token-prediction head is not included. Q4_K_M and Q8_0 were both quantized directly from
that F16 file. The vision projector was exported separately at F16. No importance matrix
(imatrix) was used.Synth was trained with thinking disabled, a short system prompt, and retrieved evidence appended to the user's final message. Use a llama.cpp build recent enough to support the Qwen3.5 architecture; the converter commit above is a known-compatible reference.
llama.cpp server (text only)
llama-server -hf Sparset/Synth-v1-qwen3.5-4b-GGUF:Q4_K_M --no-mmproj --reasoning off -c 8192 --temp 0
With the vision projector (local files)
llama-server -m Synth-v1-qwen3.5-4b-Q4_K_M.gguf --mmproj mmproj-Synth-v1-qwen3.5-4b-F16.gguf --reasoning off -c 8192 --temp 0
Flag names change between llama.cpp releases. If --reasoning off is not recognized, use your
build's option for disabling Qwen3.5 thinking (for example
--chat-template-kwargs '{"enable_thinking": false}').
Recommended message format (OpenAI-compatible /v1/chat/completions):
[
{"role": "system", "content": "Use the retrieved evidence to answer the user faithfully."},
{"role": "user", "content": "<question>\n\nRetrieved evidence:\n<evidence>"}
]
During training, evidence appeared in four formats: plain ranked chunks, Markdown sources, XML documents, and a JSON tool result. Each item carried an ID, title, source, location, and where available its authority and effective date. The source model card shows each format and a full example.
The training examples were at most 3,931 tokens, and the synthesis evaluation used a 4,096-token input budget. Longer contexts work mechanically but are untested for quality.
The source model's limitations apply. Results come from an internal synthetic evaluation and are mixed across tasks. Grounding and citations are not guaranteed. The model inherits base-model and training-data behavior, and outputs in legal, medical, financial, HR or eligibility settings need human review. Lower-bit quantization (Q4_K_M) usually loses more quality than Q8_0. For this model that loss has not been measured.
Qwen3.5-4B is released under the
Apache License 2.0. Synth v1 and these
quantizations are released for non-commercial research use only, because the training data
includes NVIDIA ChatQA's synthetic_convqa subset. NVIDIA designates that subset non-commercial
and subject to OpenAI's terms of use for generated data. See LICENSE and the
source model card.
@misc{sparset2026synth,
title = {Synth v1: A Document-Synthesis Task Model Fine-Tuned from Qwen3.5-4B},
author = {{Sparset}},
year = {2026},
howpublished = {\url{https://huggingface.co/Sparset/Synth-v1-qwen3.5-4b}}
}
Please also cite Qwen3.5 and ChatQA; the source model card has their entries.