Downloads · 30 days
146
8% of all-time downloads
starcoder2-15b-instruct-iMat.GGUF
dranger003/starcoder2-15b-instruct-iMat.GGUF
starcoder2-15b-instruct-iMat.GGUF is a text generation model from dranger003. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as bigcode-openrail-m.
<uNOTE</u: You will need a recent build of llama.cpp to run these quants (i.e. at least commit 494c870).
All-time downloads
1.8K
Public
Repo size
66.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf66.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- gguf
- License
- bigcode-openrail-m
- Access
- Public
- Created
- Mar 7, 2024
- Updated
- Mar 7, 2024
- SHA
- 63078dab
Try a prompt
Base models
Model card
From the Hugging Face model README
<u>NOTE</u>: You will need a recent build of llama.cpp to run these quants (i.e. at least commit 494c870).
GGUF importance matrix (imatrix) quants for https://huggingface.co/TechxGenus/starcoder2-15b-instruct
- The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a general purpose imatrix calibration dataset.
- The imatrix is being used on the K-quants as well.
Fine-tuned starcoder2-15b with an additional 0.7 billion high-quality, code-related tokens for 3 epochs. We used DeepSpeed ZeRO 3 and Flash Attention 2 to accelerate the training process. It achieves 77.4 pass@1 on HumanEval-Python. This model operates using the Alpaca instruction format (excluding the system prompt).
| Layers | Context | Template |
|---|---|---|
| <pre>40</pre> | <pre>16384</pre> | <pre>### Instruction<br>{instruction}<br>### Response<br>{response}</pre> |
- Task
- Text Generation
- Library
- gguf
- License
- bigcode-openrail-m
- Created
- Mar 7, 2024
- Updated
- Mar 7, 2024