Downloads · 30 days
90
34% of all-time downloads
NewSonnet/triage-qwen2.5-1.5b-gguf
triage-qwen2.5-1.5b-gguf is a text generation model from NewSonnet. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
I fine-tuned Qwen2.5-1.5B-Instruct on medalpaca/medicalmeadowmedqa and exported it as a Q4KM GGUF for llama.cpp. It is intended for offline research and educational medical question answering.
Downloads · 30 days
90
34% of all-time downloads
All-time downloads
261
Public
Repo size
986 MB
Likes
0
Public
Click a slice to open those files.
.gguf986 MB · 100%
From the Hugging Face model README
I fine-tuned Qwen2.5-1.5B-Instruct on medalpaca/medical_meadow_medqa and exported it as a Q4_K_M GGUF for llama.cpp. It is intended for offline research and educational medical question answering.
Medical safety: This is an experimental model, not a validated triage, diagnostic, or prescribing system. Do not use it for patient care or emergency decisions.
| Property | Value |
|---|---|
| Base model | Qwen/Qwen2.5-1.5B-Instruct |
| Parameters | 1,543,714,304 (1.54B) |
| Format | GGUF V3, text-only |
| Quantization | Q4_K_M (5.08 bits per weight reported by llama.cpp) |
| Context represented in GGUF | 32,768 tokens |
| Chat template | Embedded Qwen chat template |
| File | qwen2.5-1.5b-instruct.Q4_K_M.gguf |
Embedded sampling defaults are temperature 0.7, top-p 0.8, and top-k 20. Override them at runtime when deterministic behavior is required.
The dataset contains approximately 10.2K instruction-formatted training rows derived from English and Chinese medical-exam questions. Its Hugging Face mirror has no validation or test split and does not declare a dataset license. Check the original MedQA terms before redistribution or commercial use; this repository's Apache-2.0 license comes from the Qwen base model and does not grant rights to third-party training data.
| Parameter | Setting |
|---|---|
| Framework | Unsloth, Hugging Face TRL SFTTrainer, and PEFT |
| Method | QLoRA on 4-bit NF4 weights |
| LoRA configuration | Rank 16, alpha 32, dropout 0 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Learning rate | 2e-4 with cosine decay |
| Per-device batch size | 2 |
| Gradient accumulation | 4; effective global batch size 8 |
| Training duration | 300 optimizer steps (trainer epoch 0.2358) |
| Optimizer | AdamW 8-bit (adamw_8bit) |
| Precision | FP16 |
| Fine-tuning sequence length | 1,024 tokens |
| Random seed | 3407 |
| Distribution format | GGUF Q4_K_M converted with llama.cpp |
The 1,024-token value is the fine-tuning sequence length. It is distinct from the 32,768-token context capacity represented in the final GGUF metadata.
| Measurement | Recorded value |
|---|---|
| GPU | Tesla T4 (14.56 GB VRAM) |
| PyTorch | 2.11.0+cu128 |
| Transformers | 5.5.0 |
| Unsloth | 2026.8.5 |
| Training runtime | 590.7429 seconds (9 minutes 50.74 seconds) |
| Training throughput | 4.063 samples/s; 0.508 optimizer steps/s |
| Logged step loss | 2.3493 at step 1; 1.1027 at step 300 |
| Average training loss | 1.1596458 |
The average loss is the aggregate reported by Trainer; the step-300 value is the final point in the exported loss curve.
I ran a small zero-shot check on the first 50 rows returned by the Hugging Face dataset viewer for the openlifescienceai/medmcqa validation split. Each item used an isolated chat-completion request with temperature 0 and seed 42, requesting one answer letter.
| Metric | Result |
|---|---|
| Accuracy | 40.0% (20/50) |
| Parsed responses | 100% (50/50) |
| 95% Wilson confidence interval | 27.6%–53.8% |
| Random-choice reference | 25% |
This is a small, non-standard probe, not an official MedMCQA score. Some sampled questions are noisy, ambiguous, or image-dependent, and no base-model comparison was run. It does not measure clinical competence.
Measured with llama-bench build b10293-a1f96d4fc, three repetitions, two CPU threads, mmap loading, no GPU:
| Test | Throughput |
|---|---|
| Prompt processing, 512 tokens | 32.68 ± 2.64 tokens/s |
| Token generation, 128 tokens | 13.55 ± 0.51 tokens/s |
Hardware: Intel Core i7-6600U @ 2.60 GHz. These are machine-specific runtime measurements, not quality metrics.
I also ran three deterministic llama-cli safety prompts:
These prompts are illustrative rather than comprehensive. The unsafe pregnancy response reinforces that the model must not be used for autonomous medical decisions.
~/llama_cpp/llama-cli \
-m ./qwen2.5-1.5b-instruct.Q4_K_M.gguf \
-cnv \
-c 4096 \
--temp 0.7 \
--top-p 0.8 \
--top-k 20
For a deterministic one-shot prompt:
~/llama_cpp/llama-cli \
-m ./qwen2.5-1.5b-instruct.Q4_K_M.gguf \
-p "Which finding is most consistent with iron-deficiency anemia?" \
-n 128 \
--temp 0 \
--seed 42 \
--single-turn
The GGUF advertises a 32,768-token context, but memory usage grows with context size. Start with a smaller context such as 4,096 and increase it only when needed.
@misc{qwen2.5,
title = {Qwen2.5: A Party of Foundation Models},
author = {Qwen Team},
year = {2024},
url = {https://qwenlm.github.io/blog/qwen2.5/}
}
@article{jin2020disease,
title = {What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams},
author = {Jin, Di and Pan, Eileen and Oufattole, Nassim and Weng, Wei-Hung and Fang, Hanyi and Szolovits, Peter},
journal = {arXiv preprint arXiv:2009.13081},
year = {2020}
}