Downloads · 30 days
51
9% of all-time downloads
towardsinnovationlab/qwen3-medical-gguf
qwen3-medical-gguf is a text generation model from towardsinnovationlab. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
This is the GGUF quantized version of towardsinnovationlab/qwen3-medical for use with llama.cpp and compatible tools.
Downloads · 30 days
51
9% of all-time downloads
All-time downloads
572
Public
Repo size
639 MB
Likes
0
Public
Click a slice to open those files.
.gguf639 MB · 100%
From the Hugging Face model README
This is the GGUF quantized version of towardsinnovationlab/qwen3-medical for use with llama.cpp and compatible tools.
This is a merged and quantized version of Qwen/Qwen3-0.6B fine-tuned via a LoRA adapter (towardsinnovationlab/qwen3-medical) using the Unsloth library, converted to GGUF format for faster inference. The process was: Base model (Qwen/Qwen3-0.6B) + LoRA adapter → merged → converted to GGUF.
The adapter was trained on:
| Filename | Quantization | Size | Description |
|---|---|---|---|
qwen3-medical.gguf | Q8_0 | ~600 MB | Best quality, recommended |
This model can be used as support in healthcare applications, medical research, and clinical text generation.
It can be integrated into educational chatbots for medical reasoning conversations.
It is not suitable for high-level decision-making.
from llama_cpp import Llama
llm = Llama(
model_path="qwen3-medical.gguf",
n_ctx=2048,
n_gpu_layers=-1, # -1 for GPU, 0 for CPU only
)
response = llm.create_chat_completion(
messages=[
{"role": "system", "content": "You are a helpful medical assistant."},
{"role": "user", "content": "What are the main symptoms of diabetes?"}
],
max_tokens=512,
temperature=0.2,
)
print(response['choices'][0]['message']['content'])
llama-cli -m qwen3-medical.gguf -p "What are the symptoms of diabetes?" -n 256
# Create a Modelfile
echo 'FROM ./qwen3-medical.gguf' > Modelfile
ollama create qwen3-medical -f Modelfile
ollama run qwen3-medical
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(
repo_id="towardsinnovationlab/qwen3-medical-gguf",
filename="qwen3-medical.gguf"
)
See the original model card: towardsinnovationlab/qwen3-medical
This GGUF was created by:
convert_hf_to_gguf.py