Downloads · 30 days
634
3% of all-time downloads
II-Medical-8B-1706-GGUF
Intelligent-Internet/II-Medical-8B-1706-GGUF
II-Medical-8B-1706-GGUF is a machine learning model from Intelligent-Internet. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Static quants of https://huggingface.co/Intelligent-Internet/II-Medical-8B-1706
All-time downloads
24.6K
Public
Repo size
68.8 GB
Likes
31
Public
Repo makeup
Click a slice to open those files.
.gguf68.8 GB · 100%
At a glance
- Library
- transformers
- Access
- Public
- Created
- Jun 16, 2025
- Updated
- Jun 17, 2025
- SHA
- 05931019
Base models
Model card
From the Hugging Face model README
About
Static quants of https://huggingface.co/Intelligent-Internet/II-Medical-8B-1706
Provided Quants
(sorted by size, not necessarily quality)
| Link | Type | Size/GB | Notes |
|---|---|---|---|
| GGUF | Q2_K | 3.4 | |
| GGUF | Q3_K_S | 3.9 | |
| GGUF | Q3_K_M | 4.2 | |
| GGUF | Q3_K_L | 4.5 | |
| GGUF | Q4_K_S | 4.9 | |
| GGUF | Q4_K_M | 5.1 | |
| GGUF | Q5_K_S | 5.8 | |
| GGUF | Q5_K_M | 6.0 | |
| GGUF | Q6_K | 6.8 | |
| GGUF | Q8_0 | 8.8 | |
| GGUF | F16 | 16.5 |
Running II-Medical-8B-1706
apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y
git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build \
-DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON -DLLAMA_CURL=ON
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp
./llama.cpp/llama-cli \
--model path/to/II-Medical-8B-1706.Q8_0.gguf \
--threads 32 \
--ctx-size 16384 \
--n-gpu-layers 99 \
-ot ".ffn_.*_exps.=CPU" \
--seed 3407 \
--prio 3 \
--temp 0.6 \
--min-p 0.01 \
--top-p 0.9 \
-no-cnv \
--prompt "<|im_start|>user\nI'm feeling unwell. Please help. <|im_end|>\n<|im_start|>assistant\n
- Library
- transformers
- Created
- Jun 16, 2025
- Updated
- Jun 17, 2025