l2-13b-natsuki-v0.1-qloras

l2-13b-natsuki-v0.1 QLoras:

• Various experimental LLaMA-2 13b fine tunes for Natsuki character from DDLC
• Fine-tuned on a dataset of ~800 items (dialogue scraped from game augmented by MythoMax-l2-13b to turn each into snippets of multi-turn chat dialogue between Player and Natsuki)
• Since free colab was used, QLoras could be trained and tested but not merged (more test Loras can be found here)

QLORAS

• l2-13b-natsuki-0.1 (LLaMA-2 13b chat)
• l2-13b-natsuki-0.1-MM MythoMax-L2-13b

USAGE

These are meant to be mainly for chat model with limited RP ability.

For best results: replace "Human" and "Assistant" with "Player" and "Natsuki" like so:

\nPlayer: (prompt)\nNatsuki:

Tested primarily with koboldcpp 1.41+, with q8_0 GGML, context length set to 4096, temperature at 0.7, and the rest left as is on chat mode. (No memory, world info, AN, etc. used):

koboldcpp_v1.41.exe --contextsize 4096 --lora "path/to/llama-2-13b-natsuki-v0.1-qlora/ggml-adapter-model.bin"

HYPERPARAMS

• Trained for 3 epochs
• rank: 32
• lora alpha: 64
• lora dropout: 0.5
• lr: 2e-4
• batch size: 2
• warmup ratio: 0.1
• grad steps: 4

The following bitsandbytes quantization config was used during training:

• load_in_8bit: False
• load_in_4bit: True
• llm_int8_threshold: 6.0
• llm_int8_skip_modules: None
• llm_int8_enable_fp32_cpu_offload: False
• llm_int8_has_fp16_weight: False
• bnb_4bit_quant_type: nf4
• bnb_4bit_use_double_quant: False
• bnb_4bit_compute_dtype: float16
• Framework versions: PEFT 0.4.0

WARNINGS AND DISCLAIMERS

Note that aside from formatting and other minor edits, generated portion of dataset used is mostly as is generated by LM. As such, while this version is better at coherency or chatting than previous ones, it may not reflect perfectly the characteristics of Natsuki (i.e. she may not act as Tsundere or might even agree that Manga is not literature). Next version will train on a manually curated and edited version of this dataset, where dialogue will be edited to reflect her characteristics more.

Other tests to come (i.e. fine tuning on other base models, like Airoboros or Kimiko-based model).

Finally, this model is not guaranteed to output aligned or safe outputs, use at your own risk.