Downloads · 30 days
18
2% of all-time downloads
Phi-3.5-Mini-instruct-AutoRound-4bit
kaitchup/Phi-3.5-Mini-instruct-AutoRound-4bit
Phi-3.5-Mini-instruct-AutoRound-4bit is a text generation model from kaitchup. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-4.0.
This is microsoft/Phi-3.5-mini-instruct quantized with AutoRound to 4-bit and symmetric quantization for compatibility with Marlin. The model has been created, tested, and evaluated by The Kaitchup.
All-time downloads
790
Public
Parameters
3.8B
2.3 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors2.3 GB · 100%
Parameter types
How the weights are stored.
I323.6B · 95%
Try a prompt
Model card
From the Hugging Face model README
Model Details
This is microsoft/Phi-3.5-mini-instruct quantized with AutoRound to 4-bit and symmetric quantization for compatibility with Marlin. The model has been created, tested, and evaluated by The Kaitchup.
Details on quantization process, evaluation, and how to use the model here: Fine-tuning Phi-3.5 MoE and Mini on Your Computer
- Developed by: The Kaitchup
- Language(s) (NLP): English
- License: cc-by-4.0
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- License
- cc-by-4.0
- Languages
- en
- Created
- Aug 21, 2024
- Updated
- Aug 23, 2024