Downloads · 30 days
30
7% of all-time downloads
Phi-3-mini-128k-instruct-ov-int4
jojo1899/Phi-3-mini-128k-instruct-ov-int4
Phi-3-mini-128k-instruct-ov-int4 is a text generation model from jojo1899. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This is an INT4 quantized version of the Phi-3-mini-128k-instruct model. The Python packages used in creating this model are as follows:
All-time downloads
406
Public
Repo size
6.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- phi3
- Access
- Public
- Created
- Jun 7, 2024
- Updated
- Nov 7, 2024
- SHA
- 7d937216
Try a prompt
Model card
From the Hugging Face model README
This is an INT4 quantized version of the Phi-3-mini-128k-instruct model. The Python packages used in creating this model are as follows:
openvino==2024.5.0rc1
optimum==1.23.3
optimum-intel==1.20.1
nncf==2.13.0
torch==2.5.1
transformers==4.46.2
This quantized model is created using the following command:
optimum-cli export openvino --model "microsoft/Phi-3-mini-128k-instruct" --weight-format int4 --group-size 128 --sym --ratio 1 --all-layers ./Phi-3-mini-128k-instruct-ov-int4
For more details, run the following command from your Python environment: optimum-cli export openvino --help
INFO:nncf:Statistics of the bitwidth distribution:
| Num bits (N) | % all parameters (layers) | % ratio-defining parameters (layers) |
|---|---|---|
| 4 | 100% (130 / 130) | 100% (130 / 130) |
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- License
- mit
- Created
- Jun 7, 2024
- Updated
- Nov 7, 2024