OddTheGreat_Luna-12B-V.3_EXL3

Source model

Luna-12B-V.3 by OddTheGreat

---

Provided quantized models

ExLlamaV3: v0.0.24

Requirements: A python installation with huggingface-hub module to use CLI.

Licensing

License detected: unknown

The license for the provided quantized models is inherited from the source model (which incorporates the license of its original base model). For definitive licensing information, please refer first to the page of the source or base models. File and page backups of the source model are provided below.

---

Backups

Date: 12.03.2026

Source files

<details> <summary>Source page (click to expand)</summary>

Luna_12B_V.3

This is a merge of pre-trained language models.

I'm tired of Mistral 24B, but sadly, it seems there are no consumer-grade models as good for roleplaying as it is.

However, some smaller models are worth trying. 12B Gemma is one of them.

The goal of this merge was to create a non-thinking, all-around model for everyday use, primarily on RU.

Model is capable of roleplaying and is not overly censored. (Extreme stuff wasn't tested, in my use scenarios, i haven't encountered refusals or strong indirect censorship.)

In rp, the model is able to act as an antagonist or describe harmful content.

The model works well as an assistant. I tested formatted responses, document citation, etc. (For this usage, a low temperature is better, around T0.2.)

The model is good at context attention and perceiving small details such as part numbers or dates. In a roleplaying, context attention is comparable to 24B Mistral.

RU is excellent. While RU RP was tested briefly, the assistant on RU was tested extensively. For a 12B model, the performance is great.

Of course, it's only a 12B model, with limitations and problems corresponding to its size, but for a 12B model, it works very well.

Vision is present. Works, but vision isn't my thing so i can't tell more.

I've noticed a small repetition problem when the model gives one long answer. (The context becomes poisoned by a long, monotonous answer, and because of the excellent attention, the model starts to repeat itself more and more.) It's probably fixable with a repetition penalty setting, but I haven't figured out the sweet spot where it works without breaking the response too much. On shorter answers this problem didn't occured.

Tested on the GemmaT4 preset, modified Shingane sysprompt T0.8 - 1.04 for roleplaying; a custom assistant sysprompt, T0.21 for work.

Also tested by chance on rx 6600xt and on gtx 1060 6gb (with offload ofc), it works and speed was even bearable.

</details>