Downloads · 30 days
23
0% of all-time downloads
phi-3-mini-128k-instruct-awq
jsincn/phi-3-mini-128k-instruct-awq
phi-3-mini-128k-instruct-awq is a text generation model from jsincn. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This repo includes a Version of Phi-3 that was quantized to AWQ using AutoAWQ. Currently hosting via the TGI docker image fails due to its fallback on AutoModel and that not being compatible with AWQ. Hosting on vLLM…
All-time downloads
52.1K
Public
Parameters
3.8B
2.3 GB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.safetensors2.3 GB · 100%
Parameter types
How the weights are stored.
I323.6B · 95%
Try a prompt
Model card
From the Hugging Face model README
This repo includes a Version of Phi-3 that was quantized to AWQ using AutoAWQ. Currently hosting via the TGI docker image fails due to its fallback on AutoModel and that not being compatible with AWQ. Hosting on vLLM is recommended.
To run the model you need to set the trust-remote-code (or similar) flag. While the remote code comes from microsoft (see LICENSE information in the file) you should validate the code yourself before deployment.
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- License
- mit
- Created
- May 16, 2024
- Updated
- May 19, 2024