Downloads · 30 days
2
3% of all-time downloads
Phi-3.5-mini-instruct-ov-int8
helenai/Phi-3.5-mini-instruct-ov-int8
Phi-3.5-mini-instruct-ov-int8 is a machine learning model from helenai. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
microsoft/Phi-3.5-mini-instruct converted to OpenVINO with INT8 weight compression.
All-time downloads
67
Public
Repo size
3.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin3.8 GB · 100%
At a glance
- Model type
- phi3
- Access
- Public
- Created
- Jul 2, 2025
- Updated
- Jul 2, 2025
- SHA
- 96f97d7e
Model card
From the Hugging Face model README
microsoft/Phi-3.5-mini-instruct converted to OpenVINO with INT8 weight compression.
Quick start, to download this model and run chat inference:
pip install huggingface-hub[cli] openvino-genai==2025.2
curl -O https://raw.githubusercontent.com/helena-intel/snippets/refs/heads/main/llm_chat/python/llm_chat_manual.py
huggingface-cli download helenai/Phi-3.5-mini-instruct-ov-int8 --local-dir Phi-3.5-mini-instruct-ov-int8
python llm_chat_manual.py Phi-3.5-mini-instruct-ov-int8 CPU
In the last line, change CPU to GPU to run on Intel GPU.
Model export command (to export this model yourself):
pip install --upgrade optimum-intel[openvino]
optimum-cli export openvino -m microsoft/Phi-3.5-mini-instruct --weight-format int8 Phi-3.5-mini-instruct-ov-int8
This model was exported with the following versions (from modelinfo.py)
openvino_version : 2025.2.0-19140-c01cd93e24d-releases/2025/2
nncf_version : 2.16.0
optimum_intel_version : 1.23.1
optimum_version : 1.25.3
pytorch_version : 2.7.1
transformers_version : 4.51.3
- Type
- phi3
- Created
- Jul 2, 2025
- Updated
- Jul 2, 2025