vntl-gemma2-27b-qlora

Summary

This is a Gemma 2 27B qlora trained using a slightly modified version of the VNTL-v3.1-1k dataset, concatenated with the VNTL-Chat dataset.

This was trained mostly with the same hyperparameters as the VNTL LLaMA 3 8B qlora, the differences are:

• Changed batch size and gradient accumulation from 6/5 to 15/2, this was done mostly for training efficiency since this qlora was trained using an A100.

As the 8B qlora, this version also includes the "chat mode" for Japanese grammar questions.

Translation Prompt

This is a prompt example for translation:

<<METADATA>>
[character] Name: Uryuu Sakuno (瓜生 桜乃) | Gender: Female
[character] Name: Uryuu Shingo (瓜生 新吾) | Gender: Male | Aliases: Onii-chan (お兄ちゃん)
<<TRANSLATE>>
<<JAPANESE>>
買い物に行ったっきりなかなか帰ってこないと思ったら、やっぱり、迷子になっていたらしい。
<<ENGLISH>>
I thought she was taking too long to come back from shopping, and it turns out she had gotten lost as I expected.<eos>
<<JAPANESE>>
[新吾]: 「そっか。うん、電話してみてよかった」
<<ENGLISH>>
[Shingo]: "Is that so? Well, I'm glad I called you then."<eos>
<<JAPANESE>>
彼女は基本的に落ち着いた頭のいい子なんだけど、結構な筋金の入った方向音痴でもある。
<<ENGLISH>>
She's basically a smart and level-headed girl, but she also has a very bad sense of direction.<eos>
<<JAPANESE>>
頻度こそさほどではないものの、ひとたび迷子になると、とにかく派手に道に迷ってしまう。
<<ENGLISH>>
Though not very frequently, once she gets lost, she gets lost in a spectacular way.<eos>
<<JAPANESE>>
[桜乃]: 『……ごめん』
<<ENGLISH>>
[Sakuno]: "... Sorry."<eos>
<<JAPANESE>>
[新吾]: 「ううん、こう言っちゃなんだけど、迷子でよかったよ。 桜乃は可愛いから、いろいろ心配しちゃってたんだぞ俺」
<<ENGLISH>>

The generated translation for that prompt, with temperature 0, is:

[Shingo]: "No, it's fine. I'm glad you got lost, actually. You're so cute, I was worried about you."

Chat Prompt

This is a prompt example for chat:

<<CHAT>>
<<HUMAN>>
Hi! Can you break down this sentence for me? Like, what is the meaning of "桜乃は可愛いから"?
"ううん、こう言っちゃなんだけど、迷子でよかったよ。桜乃は可愛いから、いろいろ心配しちゃってたんだぞ俺」"
<<LLM>>

The generated response for that prompt, with temperature 0, is:

Certainly! Let's start with your first question. "桜乃は可愛いから" is a phrase that consists of three parts:
1. "桜乃" is a proper noun, likely a person's name.
2. "は" is a particle that marks the topic of the sentence, in this case, "桜乃."
3. "可愛い" is an adjective that means "cute" or "adorable."
4. "から" is a conjunction that indicates a reason or cause.
So, the literal translation of "桜乃は可愛いから" would be "Because Sakura is cute." It suggests that the speaker is about to provide a reason or explanation based on the fact that the person named Sakura is cute.