Trouper-12B-GGUF

<a href="https://huggingface.co/DarwinAnim8or/Trouper-12B">Trouper-12B</a> GGUF by <a href="https://huggingface.co/DarwinAnim8or">DarwinAnim8or</a>

From the original model card:

A character roleplay model trained on the custom "Actors" dataset, fine-tuned from Mistral-Nemo-Base-12B. This model was made to expand on the things I learned from TinyRP, and to overcome certain limitations I found from it; also on an entirely new dataset made just for this model.

This model writes more naturally, less like "AI"; even more so than the 24B model I'm also releasing. I suppose this is because the 12B model saw less synthethic data, and is thus less likely to use phrases typical in AI writing & prose.

Using an importance matrix actually increases KLD (lowers quality) for Q8_0 and Q6_K, so I didn't use one for those quants. For anything smaller the importance matrix provides better quality with no inference performance penalty.

The importance matrix was computed using the Bartowski's calibration_datav3.txt dataset, which has been proven to produce good results for all kinds of tasks.

The KLD metric was measured against the wikitext-2 validation split with 4096 tokens of context.