Downloads · 30 days
16
2% of all-time downloads
phi-3-mini-4k-instruct-q4
bongodongo/phi-3-mini-4k-instruct-q4
phi-3-mini-4k-instruct-q4 is a text generation model from bongodongo. Use it when you need the model to write or continue text. It is set up for transformers.
This is a 4-bit quantized version of Phi-3 4k Instruct.
All-time downloads
681
Public
Parameters
3.9B
2.3 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors2.3 GB · 100%
Parameter types
How the weights are stored.
U83.7B · 95%
Try a prompt
Model card
From the Hugging Face model README
This is a 4-bit quantized version of Phi-3 4k Instruct.
Quantization done with:
bnb_config = BitsAndBytesConfig(
load_in_4bit = True,
bnb_4bit_use_double_quant = True,
bnb_4bit_quant_type = 'nf4',
bnb_4bit_compute_dtype = torch.bfloat16
)
model = AutoModelForCausalLM.from_pretrained(
foundation_model_name,
device_map = 'auto',
quantization_config = bnb_config,
trust_remote_code = True
)
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- Created
- Apr 28, 2024
- Updated
- Apr 28, 2024