Downloads · 30 days
177
1% of all-time downloads
MKLLM-7B-Instruct-GGUF
martinkozle/MKLLM-7B-Instruct-GGUF
MKLLM-7B-Instruct-GGUF is a machine learning model from martinkozle. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-sa-4.0.
GGUF quants of trajkovnikola/MKLLM-7B-Instruct
All-time downloads
16.7K
Public
Repo size
96.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf96.7 GB · 100%
At a glance
- License
- cc-by-nc-sa-4.0
- Access
- Public
- Created
- Jun 24, 2024
- Updated
- Jun 25, 2024
- SHA
- f50a1be5
Model card
From the Hugging Face model README
MKLLM-7B-Instruct-GGUF
GGUF quants of trajkovnikola/MKLLM-7B-Instruct
Script used
from_dir=./MKLLM-7B-Instruct
dir=./MKLLM-7B-Instruct-GGUF
base_precision=BF16
file_base=MKLLM-7B-Instruct
quants=("Q2_K" "Q3_K_S" "Q3_K_M" "Q3_K_L" "Q4_K_S" "Q4_K_M" "Q4_0" "Q4_1" "Q5_K_S" "Q5_K_M" "Q5_0" "Q5_1" "Q6_K" "Q8_0" "IQ3_XS" "IQ3_S" "IQ3_M" "IQ4_XS" "IQ4_NL")
docker run --rm -v "${from_dir}":/repo ghcr.io/ggerganov/llama.cpp:full --convert "/repo" --outtype bf16
mkdir "${dir}"
mv "${from_dir}/ggml-model-bf16.gguf" "${dir}/${file_base}-${base_precision}.gguf"
for quant in ${quants[@]};
do
echo "###########################"
echo $quant
echo "==========================="
docker run --rm -v "${dir}":/repo ghcr.io/ggerganov/llama.cpp:full --quantize "/repo/${file_base}-${base_precision}.gguf" "/repo/${file_base}-${quant}.gguf" "${quant}"
done
- License
- cc-by-nc-sa-4.0
- Languages
- mk, en
- Created
- Jun 24, 2024
- Updated
- Jun 25, 2024