Nethena-MLewd-Xwin-23B-exl2

ExllamaV2 version of the model created by Undi!

Original Model https://huggingface.co/Undi95/Nethena-MLewd-Xwin-23B

Requires ExllamaV2, which is being developed by turboderp https://github.com/turboderp/exllamav2 under an MIT license.

Main branch is 6bpw 8h (requires 25gb)

5b8h branch is 5bpw 8h (requires 22gb)

8b8h branch is 8bpw 8h (requires 30gb)

---

Undi doing chemistry again.

Layer of Xwin-Mlewd was added in a different way than I do before, result seem good, but I'm a VRAMlet so I can only run the Q2 at 2k context for now.

Need to see if it really work good or I was just lucky with my prompt.

OG model : NeverSleep/Nethena-13B

Prompt template: Alpaca

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{prompt}

### Response:

LimaRP is always kicking in and thus, this can be used to have more control on the size of the output.

Thanks Ikari.