Downloads · 30 days
17
3% of all-time downloads
pravdin/merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct-gguf
merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct-gguf is a text generation model from pravdin. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
This is a collection of GGUF quantized versions of pravdin/merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct.
Downloads · 30 days
17
3% of all-time downloads
All-time downloads
509
Public
Repo size
7.8 GB
Likes
0
Public
Click a slice to open those files.
.gguf7.8 GB · 100%
From the Hugging Face model README
This is a collection of GGUF quantized versions of pravdin/merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct.
This model was created by merging the following models:
pravdin/merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct
├── Merge Method: dare_ties
├── context-labs/meta-llama-Llama-3.2-3B-Instruct-FP16
└── unsloth/Llama-3.2-3B-Instruct
├── density: 0.6
├── weight: 0.5
Merge Method: DARE_TIES - Advanced merging technique that reduces interference between models
This repository contains multiple quantization formats optimized for different use cases:
# Download a specific quantization
wget https://huggingface.co/pravdin/merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct/resolve/main/merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct.q4_k_m.gguf
# Run with llama.cpp
./main -m merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct.q4_k_m.gguf -p "Your prompt here"
from llama_cpp import Llama
# Load the model
llm = Llama(model_path="merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct.q4_k_m.gguf")
# Generate text
output = llm("Your prompt here", max_tokens=512)
print(output['choices'][0]['text'])
# Create a Modelfile
echo 'FROM ./merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct.q4_k_m.gguf' > Modelfile
# Create and run the model
ollama create merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct -f Modelfile
ollama run merged-context-labs-meta-llama-Llama-3.2-3B-Instruct-FP16-unsloth-Llama-3.2-3B-Instruct "Your prompt here"
GGUF models are optimized for:
For best performance, ensure your hardware supports the quantization format you choose.
This model was automatically quantized using the Lemuru LLM toolkit.