vntl-7b-v0.3.1-hf-exl2

VNTL v3.5.1 EXL2 quantization branches

• main (4.0bpw)
• 5.6bpw
• 8.0bpw

original (unquantized): https://huggingface.co/lmg-anon/vntl-7b-v0.3.1-hf

---

This is a merge of the experimental VNTL v0.3.1 lora created using the VNTL-v2.5-1k dataset.

This is an prompt example:

<<START>>
Name: Uryuu Shingo (瓜生 新吾) | Gender: Male | Aliases: Onii-chan (お兄ちゃん)
Name: Uryuu Sakuno (瓜生 桜乃) | Gender: Female
<<JAPANESE>>
[桜乃]: 『……ごめん』
<<ENGLISH>> (fidelity = absolute)
[Sakuno]: 『... Sorry.』</s>
<<JAPANESE>>
[新吾]: 「ううん、こう言っちゃなんだけど、迷子でよかったよ。桜乃は可愛いから、いろいろ心配しちゃってたんだぞ俺」
<<ENGLISH>> (fidelity = high)

The generated translation for that prompt, with temperature 0, is:

[Shingo]: 「No, don't apologize. I'm just glad you're safe. You're so cute, Sakuno, I was worried sick.」