Downloads · 30 days
0
mpt-7b-gsm8k-dummy
yujiepan/mpt-7b-gsm8k-dummy
mpt-7b-gsm8k-dummy is a text generation model from yujiepan. Use it when you need the model to write or continue text. It is set up for OpenVINO.
Access
Public
Updated Feb 4, 2024
Repo size
39.9 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin13.3 GB · 33%
At a glance
- Task
- Text Generation
- Library
- OpenVINO
- Access
- Public
- Created
- Feb 1, 2024
- Updated
- Feb 4, 2024
- SHA
- 2d7044bd
Try a prompt
Model card
From the Hugging Face model README
See the benchmark scripts in this repo.
pip install deepsparse-nightly[llm]==1.6.0.20231120
pip install openvino==2023.3.0
Benchmarking
- Clone this repo
- Concatenate the big fp32 IR model:
cd ./models/neuralmagic/mpt-7b-gsm8k-pt/fp32
cat openvino_model.bin.part-a* > openvino_model.bin
- Reproduce NM paper:
deepsparse_reproduce.bash - OV benchmarkapp:
benchmarkapp_*.bash
Generating these IRs
https://github.com/yujiepan-work/24h1-sparse-quantized-llm-ov
- Task
- Text Generation
- Library
- OpenVINO
- Created
- Feb 1, 2024
- Updated
- Feb 4, 2024