Downloads · 30 days
272
6% of all-time downloads
Phi-3-medium-4k-instruct-GGUF
gaianet/Phi-3-medium-4k-instruct-GGUF
Phi-3-medium-4k-instruct-GGUF is a text generation model from gaianet. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
All-time downloads
4.6K
Public
Repo size
134 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf134 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- phi3
- Access
- Public
- Created
- May 22, 2024
- Updated
- May 22, 2024
- SHA
- bdadae1d
Base models
Model card
From the Hugging Face model README
Phi-3-medium-4k-instruct-GGUF
Original Model
microsoft/Phi-3-medium-4k-instruct
Run with Gaianet
Prompt template
prompt template: phi-3-chat
Context size
chat_ctx_size: 5120
Run with GaiaNet
-
Quick start: https://docs.gaianet.ai/node-guide/quick-start
-
Customize your node: https://docs.gaianet.ai/node-guide/customize
Quantized GGUF Models
| Name | Quant method | Bits | Size | Use case |
|---|---|---|---|---|
| Phi-3-medium-4k-instruct-Q2_K.gguf | Q2_K | 2 | 5.14 GB | smallest, significant quality loss - not recommended for most purposes |
| Phi-3-medium-4k-instruct-Q3_K_L.gguf | Q3_K_L | 3 | 7.49 GB | small, substantial quality loss |
| Phi-3-medium-4k-instruct-Q3_K_M.gguf | Q3_K_M | 3 | 6.92 GB | very small, high quality loss |
| Phi-3-medium-4k-instruct-Q3_K_S.gguf | Q3_K_S | 3 | 6.06 GB | very small, high quality loss |
| Phi-3-medium-4k-instruct-Q4_0.gguf | Q4_0 | 4 | 7.9 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
| Phi-3-medium-4k-instruct-Q4_K_M.gguf | Q4_K_M | 4 | 8.57 GB | medium, balanced quality - recommended |
| Phi-3-medium-4k-instruct-Q4_K_S.gguf | Q4_K_S | 4 | 7.95 GB | small, greater quality loss |
| Phi-3-medium-4k-instruct-Q5_0.gguf | Q5_0 | 5 | 9.62 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
| Phi-3-medium-4k-instruct-Q5_K_M.gguf | Q5_K_M | 5 | 10.1 GB | large, very low quality loss - recommended |
| Phi-3-medium-4k-instruct-Q5_K_S.gguf | Q5_K_S | 5 | 9.62 GB | large, low quality loss - recommended |
| Phi-3-medium-4k-instruct-Q6_K.gguf | Q6_K | 6 | 11.5 GB | very large, extremely low quality loss |
| Phi-3-medium-4k-instruct-Q8_0.gguf | Q8_0 | 8 | 14.8 GB | very large, extremely low quality loss - not recommended |
| Phi-3-medium-4k-instruct-f16.gguf | f16 | 16 | 27.9 GB |
Quantized with llama.cpp b2961.
- Task
- Text Generation
- Library
- transformers
- Type
- phi3
- License
- mit
- Languages
- multilingual
- Created
- May 22, 2024
- Updated
- May 22, 2024