Downloads · 30 days
193
7% of all-time downloads
Nemo-12B-Marlin-v5-GGUF
starble-dev/Nemo-12B-Marlin-v5-GGUF
Nemo-12B-Marlin-v5-GGUF is a machine learning model from starble-dev. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
[!WARNING] General Use Sampling:<br Mistral-Nemo-12B is very sensitive to the temperature sampler, try values near 0.3 at first or else you will get some weird results. This is mentioned by MistralAI at their Transfor…
All-time downloads
2.8K
Public
Repo size
77.9 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf77.9 GB · 100%
At a glance
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Aug 7, 2024
- Updated
- Aug 7, 2024
- SHA
- c44d91d6
Base models
Model card
From the Hugging Face model README
[!WARNING] General Use Sampling:<br> Mistral-Nemo-12B is very sensitive to the temperature sampler, try values near 0.3 at first or else you will get some weird results. This is mentioned by MistralAI at their Transformers section.
[!NOTE] Best Samplers:<br> I found best success using the following for Nemo-12B-Marlin-v5:<br> Temperature:
0.7-0.8<br> Top K:-1<br> Min P:0.05<br> Rep Penalty:1.03(Note, it is recommended to increase this as context length increases, I find1.10to be good at 16k+ context)
Currently this is my favorite Mistral-Nemo finetune to be released.
Original Model: UsernameJustAnother/Nemo-12B-Marlin-v5 (Thank you so much for your work ♥)
Official Quants: UsernameJustAnother/Nemo-12B-Marlin-v5-gguf (Currently only Q8_0)
How to Use: llama.cpp
Original Model License: Apache 2.0
Release Used: b3538
Quants
PPL = Perplexity, lower is better<br> Comparisons are done as QX_X Llama-3-8B against FP16 Llama-3-8B, recommended as a guideline and not as fact.
| Quant Type | Note | Size |
|---|---|---|
| Q2_K | +3.5199 ppl @ Llama-3-8B | 4.79 GB |
| Q3_K_S | +1.6321 ppl @ Llama-3-8B | 5.53 GB |
| Q3_K_M | +0.6569 ppl @ Llama-3-8B | 6.08 GB |
| Q3_K_L | +0.5562 ppl @ Llama-3-8B | 6.56 GB |
| Q4_K_S | +0.2689 ppl @ Llama-3-8B | 7.12 GB |
| Q4_K_M | +0.1754 ppl @ Llama-3-8B | 7.48 GB |
| Q5_K_S | +0.1049 ppl @ Llama-3-8B | 8.52 GB |
| Q5_K_M | +0.0569 ppl @ Llama-3-8B | 8.73 GB |
| Q6_K | +0.0217 ppl @ Llama-3-8B | 10.1 GB |
| Q8_0 | +0.0026 ppl @ Llama-3-8B | 13.00 GB |
- Library
- transformers
- License
- apache-2.0
- Created
- Aug 7, 2024
- Updated
- Aug 7, 2024