Downloads · 30 days
53
3% of all-time downloads
mixtralnt-4x7b-test-gguf
imi2/mixtralnt-4x7b-test-gguf
mixtralnt-4x7b-test-gguf is a machine learning model from imi2. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
All-time downloads
2K
Public
Repo size
117 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf117 GB · 100%
At a glance
- Access
- Public
- Created
- Dec 12, 2023
- Updated
- Dec 12, 2023
- SHA
- c2be90a0
Model card
From the Hugging Face model README
model: chargoddard/mixtralnt-4x7b-test
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate
(For You)
llm_load_print_meta: model ftype = mostly Q2_K
llm_load_print_meta: model params = 24.15 B
llm_load_print_meta: model size = 7.51 GiB (2.67 BPW)
llm_load_print_meta: model ftype = mostly Q3_K - Medium
llm_load_print_meta: model params = 24.15 B
llm_load_print_meta: model size = 9.80 GiB (3.48 BPW)
llm_load_print_meta: model ftype = mostly Q4_K - Medium
llm_load_print_meta: model params = 24.15 B
llm_load_print_meta: model size = 12.70 GiB (4.52 BPW)
llm_load_print_meta: model ftype = mostly Q5_K - Medium
llm_load_print_meta: model params = 24.15 B
llm_load_print_meta: model size = 15.49 GiB (5.51 BPW)
llm_load_print_meta: model ftype = mostly Q6_K
llm_load_print_meta: model params = 24.15 B
llm_load_print_meta: model size = 18.45 GiB (6.56 BPW)
- Created
- Dec 12, 2023
- Updated
- Dec 12, 2023