Downloads · 30 days
10
2% of all-time downloads
lemonilia/Llama-3.1-Herrsimian-8B
Llama-3.1-Herrsimian-8B is a machine learning model from lemonilia. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.1.
Downloads · 30 days
10
2% of all-time downloads
All-time downloads
529
Public
Parameters
8B
34.8 GB on disk
Likes
7
Public
Click a slice to open those files.
.safetensors17.4 GB · 100%
From the Hugging Face model README

Herrsimian is a limited-scope NSFW roleplaying model intended to replicate the writing style of a certain prolific roleplayer who used to actively participate on a few different roleplay forums until the end of 2022. It's also a test bench for studying how much mileage can be obtained with a very limited amount of high-quality training samples of consistent style—mainly by training well into the overfitting regime.
The model, finetuned on Llama-3.1-8B-base, was trained with a multi-user, multi-character paradigm, where user or model turns don't necessarily alternate as in most other instruction prompting formats. As of present however the most commonly used LLM frontends cannot readily take advantage of this feature.
Note: There is still some work to be done on the dataset to increase general usability, but prose quality should be representitative of the final model.
Notes mainly pertaining to the model trained on Llama-3.1-8B-Base.
::::user: while model turns are labeled ::::assistant:
::::user:Jack: or ::::assistant:Mark:\n\n.::::user: without any attached character name.(OOC: {{message}})<|begin_of_text|> in the case of Llama-3.1), otherwise performance will be significantly reduced.:::: or ::::user or ::::assistant can be used as stopping strings just as effectively.The BOS token was omitted here. Messages and descriptions are short for the sake of brevity. The first ::::user block works as a system instruction of some sort.
::::user: Let's engage together in a roleplay.
# Roleplay title
A new day together
# Characters
## Nanashi
Anon's long-time friend. Can be somewhat terse.
{{character description here}}
## Anon
Our main protagonist. A completely normal person.
# Scenario
After meeting together, Anon and Nanashi have a little chat.
::::user:Anon: "Hello," said Anon, "How are you?"
::::assistant:Nanashi: "Very good, thanks for asking," Nanashi responded. "What would you like to do today?"
::::user:Anon: After thinking about it for a brief while, Anon proposed, "Well, we could have a coffee, what about that?"
::::user: (OOC: Can you write in a more cheerful tone?)
::::assistant:Nanashi: "Wow, that would be great! Let's go, let's go!"
::::user:Anon: [...]
Note that in reality OOC messages might not necessarily always have the intended effect. This is something I'm looking into.
For convenience, you can use the following SillyTavern context/instruct presets. Rename the files immediately after downloading them:
There are the settings I've used for testing with text-generation-webui as the backend. Note that the DRY sampler is not yet available in Llama.cpp:
:::: as a Sequence Breaker if you use DRYAvoid using Repetition/Frequency/Presence Penalty, as they will penalize punctuation and the user/model label markers ::::.
Unsloth was used on a single RTX3090 24GB GPU with QLoRA finetuning.
As of writing, 131 manually curated, mostly human-sourced samples have been used, for a total of about 1.43M unique tokens at 12k tokens context length (2.45M tokens would have been possible with 52k+ tokens context). The preformatted data was finetuned as raw text (no masking of user turns or labels) for 5 epochs in total using a WSD scheduler without warmup, with constant learning rate for 3 epochs, followed by immediate decay to 25% of the initial learning rate (in the latest run this was simulated by resuming training at a reduced learning rate).
The learning rate was initially chosen so that the lowest eval loss on a representative long sample would occur within one epoch, and overfitting (eval loss increasing beyond the minimum point) happened after that.
In the end I opted for a LoRA Dropout rate of 0.55, which from published literature appears to be close to the optimal value.
Judging model quality in the overfitting regime is not straightforard, as it can increase up to a point and then plummet if overfitting exceeds a certain level.
max_seq_length = 12288
lora_r = 128
lora_alpha = 256
lora_dropout = 0.55
use_rslora = False
per_device_train_batch_size = 1
gradient_accumulation_steps = 1
learning_rate = 0.0000950
warmup_steps = 0
weight_decay = 0.01
num_train_epochs = 5
lr_scheduler_type = "warmup_stable_decay"
lr_scheduler_kwargs = {
'num_stable_steps': 3 epochs
'num_decay_steps': 0 steps
'min_lr_ratio': 25%
}
In the orange portion of the graph the learning rate was decreased to 25% of the initial level.
Q. Isn't the model overfitted? Isn't overfitting a bad thing?
A. By the common definition of overfitting, yes, it is overfitted. However, overfitting for chatbot use is not necessarily a drawback, see for example Zhou2023 (LIMA paper) and Ouyang2022 (GPT-3 paper), as well as the conclusions in a Guanaco replication attempt. Similarly, with Herrsimian, output quality appeared to subjectively improve with increasing number of training epochs, at least up to a point.
When perfectly fitted, the model upon testing appeared to be bland, but too much overfitting killed response creativity and diminished logic capabilities, although it made it reproduce the syle and vocabulary of the training data better (which was one of the aims of the dataset).
Q. Why raw text / text completion?
A. It's the simplest format to deal with and allows maximum freedom; additionally, the non-standard paradigm used (where user/model turns don't necessarily alternate) can make user turn masking using more complex with existing pipelines.
Q. Why didn't you use prompting format X?
A. Because the model was originally intended to employ a simple, easy-to-type prompting format that could be easily transferred to other models, without extra special tokens which can be difficult to properly train without also training the embeddings sufficiently enough. I also didn't want to use an Alpaca-like prompting format, since it conflicts with Markdown formatting.
Q. Can you train large model Y with the same data?
A. My efforts are limited to what can be done locally in reasonable amounts of time (a couple hours at most) with one 24GB GPU.
Q. Can you upload quantization Z?
A. Due to upload bandwidth limitations I have to be selective regarding what I upload onto my HuggingFace account. Additionally, GGUF quantizations of Llama 3.1 don't seem to work as intended, it turned out.
Q. What about PPO, DPO, KTO...?
A. I haven't bothered yet. Reinforcement learning from human feedback (RLHF) can decrease model creativity, anyway.
Q. Why doesn't the model contain instructions?
A. Actually, an initial iteration included multi-turn synthetic instructions to pad the data up to 1000 samples in total (similar to LIMA). I found however that they didn't have a significant positive effect on roleplaying performance. Simply overtraining on actual human roleplay (after suitable augmentation) in the target format appears to yield overall better results.
Q. What's up with the model name?
A. It's an indirect reference to the author from whom I borrowed the training data. Actually, I wanted to use a cuter name, but that might have possibly jeopardized the original author's privacy.
Q. Who is the character in the top image?
A. It's Son Biten from Touhou Juuouen - Unfinished Dream of All Living Ghost. Check out Gelbooru for the unedited source image.