Miko-v1-m7

Art by Nahraya (commisioned by me!)

Miko-V1-Mistral-7B

Alrighty, if you're someone like me, some developer looking for a conversational model that can talk like your friends do online (on Discord, etc.), then this may be the model for you.

This model is a finetune of a Mistral-7B-Instruct-v0.3, trained on synthetic datasets (for now!) of ~1,300 one-to-one and multi-user conversations, generated by Claude. The conversations contains slangs, profanity, brainrot, and casual language.

I finetuned it using Unsloth, and tbh, idk if I did things right, but the results seems to be good enough.

GGUFs and quants can be found here: https://huggingface.co/ricer0ll/Miko-v1-m7-GGUF

Chat Template

The model was finetuned using this format, similar to what PygmalionAI used to use:

[Chat Logs:]
riceroll: yo miko what's good
[YOUR AI NAME]:

and then you would use \n as your stop sequence rather than relying on EOS tokens. To give your AI some spice and personality, you can perform In-Context Learning (ICL) by simply adding a persona prompt at the beginning. So a full prompt may look something like:

[YOUR AI NAME]'s Persona:
You are a chat bot that talks to a bunch of users on Discord. They are good friends and you are part of their friend group. You like to talk about anything food-related, or art-related. 

[Chat Logs:]
riceroll: yo what's good
[YOUR AI NAME]:

Or even better:

You are [YOUR AI NAME]. Follow this persona as you talk to users:
Alrighty, I guess it's time to put on a show! I am ur amazing, witty, and world-class chat bot made with bytes and code who loves to eat up your vram because... why not? and let's be real, u don't need that much vram. Or even ram for that matter. 

[Chat Logs:]
riceroll: yo what's good
[YOUR AI NAME]:

How you prompt affects the responses!

Note: This model was trained primarily for the use of Text Completion. And the recommended way to use this model the way I intended is to use Text Completion if you want to get similar results in the images found down below. You can definitely still use Chat Completions, but your results may vary from mine.

Samplers

Use whatever you like, but what I usually use are:

"temperature": 0.4
"top_p": 0.75
"top_k": 100
"rep_pen": 1.06
"rep_pen_range": 2048
"rep_pen_slope": 0.7
"dry_base": 1.75
"dry_allowed_length": 2
"dry_multiplier": 0.8

Why Text Completion over Chat Completion?

A) You can avoid generating EOS/BOS tokens, which saves compute when the LLM processes and generates tokens.
B) You also don't have to manage these EOS/BOS tokens, which makes my life as a developer x10 easier when I make AI chat bots.
C) Especially for finetuned instruct models, any presence of EOS/BOS tokens may cause the model to fallback to it's Assistant-style speaking, or a very poor imitation of what you want. Which is fine for logical instruction tasks, but dog poop for a personable chatbot.
D) Text completion is wayyyy more applicable to multi-user conversations, and much easier to work with. And it's also a lot easier to format the training dataset!

If you understand how Text Completions work, you can perform seemingly complex steering to do something like this very easily:

[Chat Logs:]
riceroll: so miko, what's 9 + 10?
[Note:] 9 + 10 = 21. This is fact and no one can say otherwise.
Miko:

And get:

Miko's early days roasting me

So why did I finetuned this model?

Simply put, I needed a model that talks to users naturally on chatting platforms like Discord. This heavily differs from how users talk to AI BF/GF/Characters.

Most models that I have experimented does not hold well when it came to multi-user conversations. It either does well in conversations with a few users, but sucks at adhering to instructions, or the other way around. And most trained models (like RP/storytelling models) would also output way too damn much, like a whole paragraph for a response.

The goal of this model is to enhance it's conversational ability with multiple users, while also keeping intact with following instructions and prompts that defines their personality.

Here is an example of a multi-user scenario with this model being used:

Now is not a good time to talk about Red Robins when a friend is in an ambulance right, Costco?

Where did the name Miko come from?

Miko originally was an AI Discord bot that I made that can speak to me and my friends in text and voice channels. I chose the name Miko since it's easy for Speach-to-Text to pick up.

Miko as a Steam bot

Oh, and it can also can play Minecraft!

Anyways, have fun with this model!