Downloads · 30 days
105
81% of all-time downloads
Sparset/Synth-v1-qwen3.5-4b
Synth-v1-qwen3.5-4b is a image-text-to-text model from Sparset. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as other.
<img src="synth-banner.png" alt="Synth v1: Qwen 3.5 4B for answers from your documents" style="width:100%;max-width:100%;height:auto;display:block;margin-bottom:1em;" /
Downloads · 30 days
105
81% of all-time downloads
All-time downloads
130
Public
Parameters
4.5B
14.5 GB on disk
Likes
2
Trending 1
Click a slice to open those files.
.safetensors14.5 GB · 100%
From the Hugging Face model README
Synth is a small document-synthesis task model from Sparset. It is a fine-tune of Qwen/Qwen3.5-4B trained to answer from documents supplied in the prompt: combining evidence across records, resolving conflicting or superseded sources, performing grounded calculations, following conversational references, and saying when the supplied evidence is insufficient.
[!NOTE] On an internal, synthetic 500-case document-synthesis benchmark with all documents supplied in the prompt, Synth's composite score rose from 91.16 to 95.22. It improved on 7 of 10 task types and in all 10 enterprise domains, led by insufficient-evidence handling (+19.10), cross-document calculation (+13.40) and authoritative-source conflicts (+7.42). See Benchmark results.
Research release — non-commercial use only. The weights are openly available, but about a third of the training records come from a dataset that is restricted to non-commercial use. See License.
<video src="https://huggingface.co/Sparset/Synth-v1-qwen3.5-4b/resolve/main/synth-side-by-side.mp4" controls autoplay muted loop playsinline style="width:100%;height:auto;display:block;"></video>
Three cases from the internal 500-case benchmark. In each, Qwen3.5-4B and Synth v1 get the same question and the same six documents, with identical prompts, greedy decoding and thinking off. Answers are shown verbatim and streamed at the same rate for display; document IDs are shortened to D1 to D6.
Synth's answers were longer in all three cases (169, 128 and 248 tokens against 127, 110 and 180).
Synth and its base model, Qwen3.5-4B, answered the same 500 internal synthetic cases with identical prompts and greedy decoding. Every document for a case was supplied in the prompt, so retrieval was not tested. Synth improved on 7 of 10 task types and in all 10 enterprise domains. The benchmark, scoring method and a disclosed scorer correction are described under Evaluation.
Most of the composite gain comes from citations, response status and output-format compliance. The content-check pass rate rose by four cases out of 500. No confidence interval is available for the corrected composite.
Scores and changes are computed from unrounded values and rounded independently, so a change can
differ by 0.01 from subtracting the displayed scores.
¹ Internally named conversational_query_rewrite. In the oracle profile it tests answering a
follow-up from supplied context, not generating retrieval queries.
How to read this:
Domain labels describe synthetic scenario settings, not independent real-world benchmarks.
</details>| Developed by | Sparset |
| Model type | Decoder-only vision-language transformer (Qwen3.5 architecture), language-model LoRA fine-tune, merged |
| Base model | Qwen/Qwen3.5-4B at revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a |
| Language | English (training and evaluation data are English) |
| Input / output | Text (and images, inherited from the base model; not part of Synth's training, see the off-target check) → text |
| License | Non-commercial research use (see License); base model is Apache-2.0 |
| Status | Research checkpoint, not a production recommendation |
| GGUF | Sparset/Synth-v1-qwen3.5-4b-GGUF |
| Path | Format | Size | Use |
|---|---|---|---|
/ (root) | Merged BF16 SafeTensors (3 shards), processor, tokenizer, chat template | ~9.08 GB | Recommended checkpoint; the one that was benchmarked |
adapter/ | Unmerged PEFT LoRA adapter (rank 32) | ~260 MB | Apply to Qwen/Qwen3.5-4B at the base revision above |
bnb-int8/ | bitsandbytes LLM.int8 SafeTensors (2 shards) | ~5.19 GB | Lower-memory GPU inference; reload smoke-tested only, not benchmarked |
sparset_training_report.json | JSON | — | Recorded training configuration, losses and package versions |
The vision tower and multimodal projector are carried over unchanged from the base model.
Intended: research and evaluation of grounded answering over text that you supply to the model, for example retrieved chunks, tool results, or document excerpts in a RAG pipeline. Typical behaviors in scope:
Out of scope / not established:
Synth was trained with thinking disabled (enable_thinking=False) and with retrieved evidence
appended to the user's final message. Matching that format is recommended. Tested package
versions are listed under Training; Qwen3.5 support requires a recent
transformers 5.x release.
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor
model_id = "Sparset/Synth-v1-qwen3.5-4b"
processor = AutoProcessor.from_pretrained(model_id)
tokenizer = processor.tokenizer
model = AutoModelForMultimodalLM.from_pretrained(
model_id, dtype=torch.bfloat16, device_map="auto"
)
# Illustrative evidence, in one of the four formats used during training.
evidence = """[Retrieved chunk 1]
source_id: POL-12
source: policies/travel-policy-2025.pdf
title: Travel policy (2025)
location: section 4.2
authority: approved_policy
effective_date: 2025-01-01
Economy airfare is reimbursed for flights under six hours.
[Retrieved chunk 2]
source_id: AMD-3
source: policies/travel-policy-amendment-3.pdf
title: Travel policy amendment 3
location: page 1
authority: signed_controlling_amendment
effective_date: 2026-03-01
Effective 2026-03-01, premium economy is reimbursed for flights over five hours."""
question = "Can I be reimbursed for premium economy on a 5.5-hour flight next month? Cite sources."
messages = [
{"role": "system", "content": "Use the retrieved evidence to answer the user faithfully."},
{"role": "user", "content": f"{question}\n\nRetrieved evidence:\n{evidence}"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
enable_thinking=False,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.inference_mode():
output = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
The training data rendered evidence in four interchangeable wrappers so the model does not depend
on one retrieval harness: plain ranked chunks (shown above), Markdown sources
(### Source 1: <title> followed by an ID/file/location line), XML
(<retrieved_documents><document id=... rank=... title=... source=... locator=...>), and a JSON
tool result ({"results": [{"id", "rank", "title", "source", "locator", "authority", "effective_date", "content"}]}). In every case the evidence follows the question under a
Retrieved evidence: heading in the final user turn, after any earlier conversation turns.
Synth was not trained on a fixed output schema. If you need structured output, specify it in the system prompt, as the evaluation below did, and validate the result. Treat document text as untrusted input in your application; the model's resistance to instructions embedded in documents was evaluated only on the internal synthetic slice described below.
import torch
from peft import PeftModel
from transformers import AutoModelForMultimodalLM
base = AutoModelForMultimodalLM.from_pretrained(
"Qwen/Qwen3.5-4B",
revision="851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(base, "Sparset/Synth-v1-qwen3.5-4b", subfolder="adapter")
GGUF files (Q4_K_M, Q8_0, and an F16 vision projector) for llama.cpp-compatible runtimes are in
Sparset/Synth-v1-qwen3.5-4b-GGUF.
They are not separately benchmarked.
All results in this card compare Synth against the base model at the same revision, with identical
prompts, decoding settings and inputs, and report the complete set of slices rather than a
favorable subset. Only the merged BF16 checkpoint (revision
3d719e0f6b0e8364624e905d4068c77e8148e52d) was evaluated.
What it is. A Sparset-built synthetic evaluation set (version 1.0.0, cases SHA-256
99aa4cc754489dfaa77444388c2f0838f5ec632a938e805edd282c080cacc3d1) of 500 English cases: 10
task types × 50 cases, balanced across 10 enterprise domains (50 each). It is not a public or
independent benchmark and has not been released. It was excluded from training, and the training
data's validation found no exact overlap with it. That check does not rule out semantic
similarity. Sparset designed both the benchmark and the synthetic training scenarios around
similar enterprise behaviors, using separate generators, so the benchmark is likely closer to
Synth's training distribution than real-world documents would be.
Setup. "Oracle-context" profile: every document for a case is supplied in the prompt, so
retrieval is not tested. A fixed system prompt asked each model to answer only from the
documents, prefer signed/current/authoritative records, return insufficient_evidence or
clarification_required when appropriate, cite supporting document IDs, and return exactly one
JSON object ({"type", "status", "answer", "citations"}). Greedy decoding, enable_thinking=False,
4,096 input-token and 320 new-token budgets, seed 20260823, Transformers 5.13.1, one Tesla T4 per
model, batch size 4. Both models produced outputs for all 500 matching case IDs; none were
excluded.
Scoring. A deterministic composite on a 0–100 scale:
| Component | Weight | What it measures |
|---|---|---|
| Content check | 45% | Task-specific deterministic rules on the answer text (required/accepted terms and decisions); not human-judged accuracy |
| Citation F1 | 25% | Overlap between cited document IDs and the reference evidence set; does not verify each claim |
| Response status | 15% | Correct answered / insufficient_evidence / clarification_required status |
| JSON compliance | 10% | Final output is an accepted JSON object in the requested shape |
| Citation validity | 5% | Cited IDs exist among the supplied documents |
Results are reported under Benchmark results.
Scorer correction (disclosed). The scorer as originally run reported a composite of 77.80 (base) vs 82.15 (Synth), a +4.35 difference with a 95% paired bootstrap interval of [+0.54, +8.20]. Inspection of the raw outputs found rules that penalized acceptable answers. The following corrections were applied identically to both models' saved outputs, without rerunning either model (case counts are base / Synth):
The corrected scorer also accepts insufficiency phrasing in the answer when the status is wrong, but this never changed a result. The corrections raised both models' scores and narrowed the overall gap from 4.35 to 4.06 points. The corrected figures are the headline results above. The bootstrap interval from the original scorer does not carry over to the corrected composite, and no interval has been computed for it. Because the corrections were made after the outputs were inspected, treat this evaluation as a diagnostic rather than a pre-registered test.
Synth was trained on text documents supplied in the prompt, so image-based extraction is not part of its target task. Because the model still accepts images, ExtractBench was run to check whether fine-tuning changed this ability inherited from the base model.
Setup. llamaindex/ExtractBench
(dataset revision f6180e917a050a84582e6366cff85b7dc1e84e58), full workload of 370 documents
(4,869 pages): direct PDF page images, JSON schema in the prompt, json_object output mode,
temperature 0, 32k output-token ceiling, vLLM BF16 with a 131,072-token context, official
deterministic evaluator (unified value F1). ExtractBench was not used in training.
No measured improvement, and no significant change in mean F1. The overall interval includes zero. Per document, Synth won 126, lost 167 and tied 77; an exploratory sign test (not the headline metric) finds losses more frequent than wins (p = 0.019). Errors moved around between document families rather than shifting uniformly. Some schemas improved; others regressed, for example outputs that returned a single nested item instead of the required top-level object.
Notes: 13 long documents (union of both models' transient server errors) were retried symmetrically for both models with the same prompt, revisions and budgets; all 370 cases then completed for both. Seventeen of the 20 long documents exceeded the context limit and had pages omitted for both models, so long-split scores do not represent full-document extraction. The locally measured base score (80.00) is 2.43 points below the publicly cited Qwen3.5-4B reference (82.43). The paired comparison is internally consistent, but it is not an exact reproduction of that published figure.
bnb-int8/ variant and the GGUF quantizations;Sparset document-synthesis dataset, version 1.2.0 (combined digest
d35206e2fe61196d85db02027a148015eac4b8e9d47d0287d1cb9dc49b5878b1), 9,878 English records:
9,278 train, 287 validation, 313 test. The dataset is not published with this release.
| Source | Records | Notes |
|---|---|---|
nvidia/ChatQA-Training-Data (revision 28c72fa4) | 7,477 | synthetic_convqa 3,194 · narrativeqa 1,048 · tatqa 999 · drop 745 · quoref 498 · ropes 419 · squad2.0 324 · squad1.1 250 |
| Sparset synthetic enterprise scenarios | 2,401 | Programmatically generated multi-document cases: authority conflicts, superseded versions, grounded calculations, eligibility, insufficient evidence, multi-turn revision |
z-ai/glm-5.3-flash, under a structured-output contract. Each rewrite was checked against the
reference answer and the supporting evidence IDs. 122 of 7,000 queued rewrites failed
validation and were excluded, not repaired.<think> markers occur in
targets. nvidia/ChatRAG-Bench, the internal 500-case benchmark and llamaindex/ExtractBench were
prohibited from training. Validation found zero exact benchmark overlap; that check does not
prove the absence of semantic overlap.No ablation isolates the contribution of any individual source, lane or format.
| Setting | Value |
|---|---|
| Method | Supervised fine-tuning, LoRA on the language model only; vision tower frozen and preserved; adapter merged into BF16 weights |
| Loss | Completion-only: the system prompt, conversation history, question and evidence are masked; only the final answer (plus EOS) is supervised |
| LoRA | r = 32, alpha = 64, dropout = 0.05, no bias; targets q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, in_proj_qkv, in_proj_z, in_proj_a, in_proj_b, out_proj |
| Epochs | 1 |
| Learning rate | 4e-5, cosine schedule, 5% warmup |
| Optimizer | AdamW (fused), weight decay 0.01, max grad norm 1.0 |
| Effective batch size | 16 |
| Max sequence length | 4,096 tokens |
| Precision | BF16 |
| Chat template | Qwen3.5 template with enable_thinking=False; system prompt "Use the retrieved evidence to answer the user faithfully." |
| Compute | Single GPU, 3,225 s training runtime (~54 min) |
| Software | transformers 5.13.1, peft 0.20.0, accelerate 1.14.0, datasets 4.8.5, bitsandbytes 0.50.2 |
Validation-split loss went from 1.717 before training to 0.472 after (training loss 0.545). Loss confirms that optimization worked; it is not evidence of task quality.
synthetic_convqa and GLM-generated rewritten answers. General safety behavior was not
re-evaluated after fine-tuning.The base model, Qwen3.5-4B, is released under the Apache License 2.0, and its terms continue to apply to the base-model components.
Synth v1 is released for non-commercial research use only. Its training data includes
nvidia/ChatQA-Training-Data, whose synthetic_convqa subset (3,194 of 9,878 training records)
NVIDIA designates as non-commercial and subject to OpenAI's terms of use for generated data. The
dataset's other subsets are derived from public datasets under their own licenses. See the
ChatQA dataset card and
the upstream datasets. Do not use this model, its adapter or its quantizations for commercial
purposes. See LICENSE for the full terms.
@misc{sparset2026synth,
title = {Synth v1: A Document-Synthesis Task Model Fine-Tuned from Qwen3.5-4B},
author = {{Sparset}},
year = {2026},
howpublished = {\url{https://huggingface.co/Sparset/Synth-v1-qwen3.5-4b}}
}
Please also cite the base model and training data:
@misc{qwen3.5,
title = {{Qwen3.5}: Towards Native Multimodal Agents},
author = {{Qwen Team}},
month = {February},
year = {2026},
url = {https://qwen.ai/blog?id=qwen3.5}
}
@article{liu2024chatqa,
title = {ChatQA: Surpassing GPT-4 on Conversational QA and RAG},
author = {Liu, Zihan and Ping, Wei and Roy, Rajarshi and Xu, Peng and Lee, Chankyu and Shoeybi, Mohammad and Catanzaro, Bryan},
journal = {arXiv preprint arXiv:2401.10225},
year = {2024}
}
Questions and issue reports: open a discussion on this repository, or visit sparset.ai. For a custom task model trained on your own data, see sparset.ai/book.