Downloads · 30 days
106
5% of all-time downloads
dolphin-2.9.4-llama3.1-8b-GGUF
NeoChen1024/dolphin-2.9.4-llama3.1-8b-GGUF
dolphin-2.9.4-llama3.1-8b-GGUF is a machine learning model from NeoChen1024. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Perplexity measured with -fa -c 2048 -ub 2048 on UTF-8 text version of "Wired Love" from Project Gutenberg.
All-time downloads
2.3K
Public
Repo size
50.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf50 GB · 100%
At a glance
- Access
- Public
- Created
- Sep 25, 2024
- Updated
- Sep 25, 2024
- SHA
- ca73e4f8
Base models
Model card
From the Hugging Face model README
GGUF quants of cognitivecomputations/dolphin-2.9.4-llama3.1-8b, here I have:
IQ4_XS (4.2G, 8.7992 +/- 0.11237, fits into 8GiB VRAM + 4096 context with F16 KV cache)
Q4_K_M (4.6G, 8.7948 +/- 0.11223, fits into 8GiB VRAM + 4096 context with F16 KV cache, also good for CPU inference on E5-26xx v3/v4)
Q8_0 (8.0G, 8.5970 +/- 0.10933, imatrix derived from it)
F16 ( 15G, 8.6617 +/- 0.11043, for 24GiB VRAM)
Perplexity measured with -fa -c 2048 -ub 2048 on UTF-8 text version of "Wired Love" from Project Gutenberg.
- Created
- Sep 25, 2024
- Updated
- Sep 25, 2024