Downloads · 30 days
13
9% of all-time downloads
phi-4-4bit-autoround-bnb
fhamborg/phi-4-4bit-autoround-bnb
phi-4-4bit-autoround-bnb is a text generation model from fhamborg. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
[](https://huggingface.co/fhamborg/phi-4-4bit-gptq)
All-time downloads
144
Public
Parameters
2.9B
9.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors9.1 GB · 100%
Parameter types
How the weights are stored.
I321.7B · 60%
Try a prompt
Base models
Model card
From the Hugging Face model README
Phi-4 GPTQ (4-bit Quantized)
Model Description
This is a 4-bit quantized version of the Phi-4 transformer model, optimized for efficient inference while maintaining performance.
- Base Model: Phi-4
- Quantization: autoround and bnb (4-bit)
- Format:
safetensors - Tokenizer: Uses standard
vocab.jsonandmerges.txt
Intended Use
- Fast inference with minimal VRAM usage
- Deployment in resource-constrained environments
- Optimized for low-latency text generation
Model Details
| Attribute | Value |
|---|---|
| Model Name | Phi-4 GPTQ |
| Quantization | 4-bit (GPTQ) |
| File Format | .safetensors |
| Tokenizer | phi-4-tokenizer.json |
| VRAM Usage | ~X GB (depending on batch size) |
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- License
- mit
- Languages
- en
- Created
- Feb 20, 2025
- Updated
- Feb 20, 2025