Downloads · 30 days
8
4% of all-time downloads
Phi-3-mini-4k-instruct-onnx-cpu-int4-rtn-block-32
EmbeddedLLM/Phi-3-mini-4k-instruct-onnx-cpu-int4-rtn-block-32
Phi-3-mini-4k-instruct-onnx-cpu-int4-rtn-block-32 is a text generation model from EmbeddedLLM. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
We measured the performance of CPU-INT4-RTN-BLOCK-32 on AMD Ryzen 9 7940HS /w Radeon 78
All-time downloads
208
Public
Repo size
2.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.data2.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- phi3
- Access
- Public
- Created
- Jul 18, 2024
- Updated
- Jul 19, 2024
- SHA
- 68f7497f
Try a prompt
Model card
From the Hugging Face model README
EmbeddedLLM/Phi-3-mini-4k-instruct-onnx-cpu-int4-rtn-block-32
Performance Metrics
<!-- These are the evaluation metrics being used, ideally with a description of why. -->CPU-INT4-RTN-BLOCK-32
We measured the performance of CPU-INT4-RTN-BLOCK-32 on AMD Ryzen 9 7940HS /w Radeon 78
| Prompt Length | Generation Length | Average Throughput (tps) |
|---|---|---|
| 128 | 128 | - |
| 128 | 256 | - |
| 128 | 512 | - |
| 128 | 1024 | - |
| 256 | 128 | - |
| 256 | 256 | - |
| 256 | 512 | - |
| 256 | 1024 | - |
| 512 | 128 | - |
| 512 | 256 | - |
| 512 | 512 | - |
| 512 | 1024 | - |
| 1024 | 128 | - |
| 1024 | 256 | - |
| 1024 | 512 | - |
| 1024 | 1024 | - |
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- License
- mit
- Languages
- en
- Created
- Jul 18, 2024
- Updated
- Jul 19, 2024