Downloads · 30 days
22
9% of all-time downloads
Phi-3.5-mini-instruct-ov-int4
jojo1899/Phi-3.5-mini-instruct-ov-int4
Phi-3.5-mini-instruct-ov-int4 is a machine learning model from jojo1899. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This is an INT4 quantized version of the Phi-3.5-mini-instruct model. The Python packages used in creating this model are as follows:
All-time downloads
232
Public
Repo size
2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin2 GB · 100%
At a glance
- License
- mit
- Model type
- phi3
- Access
- Public
- Created
- Nov 7, 2024
- Updated
- Nov 7, 2024
- SHA
- 7ce33105
Model card
From the Hugging Face model README
This is an INT4 quantized version of the Phi-3.5-mini-instruct model. The Python packages used in creating this model are as follows:
openvino==2024.5.0rc1
optimum==1.23.3
optimum-intel==1.20.1
nncf==2.13.0
torch==2.5.1
transformers==4.46.2
This quantized model is created using the following command:
optimum-cli export openvino --model "microsoft/Phi-3.5-mini-instruct" --weight-format int4 --group-size 128 --sym --ratio 1 --all-layers ./Phi-3.5-mini-instruct-ov-int4
For more details, run the following command from your Python environment: optimum-cli export openvino --help
INFO:nncf:Statistics of the bitwidth distribution:
| Num bits (N) | % all parameters (layers) | % ratio-defining parameters (layers) |
|---|---|---|
| 4 | 100% (130 / 130) | 100% (130 / 130) |
- Type
- phi3
- License
- mit
- Created
- Nov 7, 2024
- Updated
- Nov 7, 2024