Downloads · 30 days
34
19% of all-time downloads
PocketWeights-Qwen2.5-7B-Instruct
PocketWeights/PocketWeights-Qwen2.5-7B-Instruct
PocketWeights-Qwen2.5-7B-Instruct is a machine learning model from PocketWeights. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
Heavy models, made light. PocketWeights specializes in targeted weight synthesis and hardware-friendly deployments for local AI practitioners.
All-time downloads
180
Public
Repo size
19 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf19 GB · 100%
At a glance
- Library
- gguf
- License
- apache-2.0
- Access
- Public
- Created
- Sep 6, 2026
- Updated
- Sep 7, 2026
- SHA
- 1be5b7e4
Base models
Model card
From the Hugging Face model README
⚡ PocketWeights: Qwen2.5-7B-Instruct (GGUF)
Heavy models, made light. PocketWeights specializes in targeted weight synthesis and hardware-friendly deployments for local AI practitioners.
Synthesized via out-of-core DARE-TIES weight merging and quantized through llama.cpp.
🧠 Model Pedigree
- Primary Base:
Qwen/Qwen2.5-7B-Instruct - Secondary Alignment:
Qwen/Qwen2.5-Coder-7B-Instruct - Synthesis Engine: MergeKit (DARE-TIES)
📊 Hardware & VRAM Compatibility Guide
| File | Format | Target VRAM | Best For Hardware |
|---|---|---|---|
Qwen2.5-7B-Instruct-Q4_K_M.gguf | 4-bit Medium | ~5.5 – 6.5 GB | Consumer GPUs (RTX 3060/4060), 8GB Macs, CPU offload |
Qwen2.5-7B-Instruct-Q6_K.gguf | 6-bit | ~7.5 – 8.5 GB | Near-lossless instruction precision, 10GB+ GPUs |
Qwen2.5-7B-Instruct-Q8_0.gguf | 8-bit | ~9.0 – 10.5 GB | Highest numerical precision, 12GB GPUs, system RAM inference |
🚀 Quickstart (Ollama)
Run directly from Hugging Face without manual downloads:
# 4-bit
ollama run hf.co/PocketWeights/PocketWeights-Qwen2.5-7B-Instruct:Qwen2.5-7B-Instruct-Q4_K_M
# 6-bit
ollama run hf.co/PocketWeights/PocketWeights-Qwen2.5-7B-Instruct:Qwen2.5-7B-Instruct-Q6_K
# 8-bit
ollama run hf.co/PocketWeights/PocketWeights-Qwen2.5-7B-Instruct:Qwen2.5-7B-Instruct-Q8_0
🤝 Support the PocketWeights Mission
I build, verify, and publish custom weight merges and quantization pipelines to provide high-quality, unrestricted, and hardware-friendly models to the open-source community for free.
Running conversion setups, cloud instances, and storage requires ongoing compute resources. If these models enhance your local workflow, save you API costs, or power your projects, consider supporting ongoing pipelines:
☕ Donation Options
- Ko-fi: ko-fi.com/iamvishalnarayan
- Web3 / Crypto (Polygon / ETH):
0x4FC189bf839A89259dd28DE8cD97883c49e15615
Tip: Transferring over the Polygon network keeps transaction gas fees below $0.01!
- Library
- gguf
- License
- apache-2.0
- Created
- Sep 6, 2026
- Updated
- Sep 7, 2026