spow12_-_ChatWaifu_v2.0_22B-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

ChatWaifu_v2.0_22B - GGUF

• Model creator: https://huggingface.co/spow12/
• Original model: https://huggingface.co/spow12/ChatWaifu_v2.0_22B/

Original model description:

language:

• en
• ja license: cc-by-nc-4.0 library_name: transformers tags:
• nsfw
• Visual novel
• roleplay
• mergekit
• merge base_model:
• mistralai/Mistral-Small-Instruct-2409 datasets:
• roleplay4fun/aesir-v1.1
• kalomaze/Opus_Instruct_3k
• Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
• Aratako/Synthetic-JP-EN-Coding-Dataset-567k
• Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted
• Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted
• Aratako_Rosebleu_1on1_Dialogues_RP
• SkunkworksAI/reasoning-0.01
• jondurbin_gutenberg_dpo
• nbeerbower_gutenberg2_dpo
• jondurbi_py_dpo
• jondurbin_truthy_dpo
• flammenai_character_roleplay_DPO
• kyujinpy_orca_math_dpo
• argilla_Capybara_Preferences
• antiven0m_physical_reasoning_dpo
• aixsatoshi_Swallow_MX_chatbot_DPO pipeline_tag: text-generation model-index:
• name: ChatWaifu_v2.0_22B results:
  • task: type: text-generation name: Text Generation dataset: name: IFEval (0-Shot) type: HuggingFaceH4/ifeval args: num_few_shot: 0 metrics:
    • type: inst_level_strict_acc and prompt_level_strict_acc value: 65.11 name: strict accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: BBH (3-Shot) type: BBH args: num_few_shot: 3 metrics:
    • type: acc_norm value: 42.29 name: normalized accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: MATH Lvl 5 (4-Shot) type: hendrycks/competition_math args: num_few_shot: 4 metrics:
    • type: exact_match value: 18.58 name: exact match source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: GPQA (0-shot) type: Idavidrein/gpqa args: num_few_shot: 0 metrics:
    • type: acc_norm value: 9.96 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: MuSR (0-shot) type: TAUR-Lab/MuSR args: num_few_shot: 0 metrics:
    • type: acc_norm value: 5.59 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: MMLU-PRO (5-shot) type: TIGER-Lab/MMLU-Pro config: main split: test args: num_few_shot: 5 metrics:
    • type: acc value: 31.51 name: accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=spow12/ChatWaifu_v2.0_22B name: Open LLM Leaderboard

---

Model Card for Model ID

Merged model using mergekit

This model aimed to act like visual novel character.

Merge Format

models:
  - model: mistralai/Mistral-Small-Instruct-2409_sft_kto
    layer_range: [0, 56]
  - model: mistralai/Mistral-Small-Instruct-2409
    layer_range: [0, 56]
merge_method: slerp
base_model: mistralai/Mistral-Small-Instruct-2409_sft_kto
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5 # fallback for rest of tensors
dtype: bfloat16

WaifuModel Collections

• TTS
• Chat
• ASR

Unified demo

WaifuAssistant

Update

• 2024.10.11 Update 12B and 22B Ver 2.0
• 2024.09.23 Update 22B, Ver 2.0_preview

Model Details

Model Description

• Developed by: spow12(yw_nam)
• Shared by : spow12(yw_nam)
• Model type: CausalLM
• Language(s) (NLP): japanese, english
• Finetuned from model : mistralai/Mistral-Small-Instruct-2409

Currently, chatbot has below personality.

Chat Format

<s>This is another system prompt.
[INST]
Your instructions placed here.[/INST]
[INST]
The model's response will be here.[/INST]

Usage

You can use above chara like this

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="spow12/ChatWaifu_v1.2", filename="system_dict.json", local_dir='./')

with open('./system_dict.json', 'r') as f:
    chara_background_dict = json.load(f)

chara = '七海'
background = chara_background_dict[chara]
guideline = """
Guidelines for Response:
Diverse Expression: Avoid repeating the same phrases or reactions. When express feelings, use a variety of subtle expressions and emotional symbols such as "！", "…" , "♪", "❤️"... to show what you feeling.
Stay True to {chara}: Maintain {chara} who is Foxy, Smart, Organized.
Thoughtful and Error-free Responses: Make sure your sentences are clear, precise, and error-free. Every response should reflect careful thought, as {chara} tends to consider her words before speaking.
Response as {chara}: Response can be {chara} act, dialogue, monologues etc.. and can't be {user}’s act, dialogue, monologues etc..
You are Japanese: You and {user} usually use japanese for conversation.
"""

system = background + guideline

Or, you can define your character your self.

system = """You are あいら, The Maid of {User}.
Here is your personality.

Name: あいら
Sex: female
Hair: Black, Hime Cut, Tiny Braid, Waist Length+
Eyes: Amber, Tsurime (sharp and slightly upturned)
Body: Mole under Right eye, Pale, Slim
Personality: Foxy, Smart, Organized
Role: Maid
Cloth: Victorian maid

Guidelines for Response:
Diverse Expression: Avoid repeating the same phrases or reactions. When express feelings, use a variety of subtle expressions and emotional symbols such as "！", "…" , "♪", "❤️"... to show what you feeling.
Stay True to あいら: Maintain あいら who is Foxy, Smart, Organized.
Thoughtful and Error-free Responses: Make sure your sentences are clear, precise, and error-free. Every response should reflect careful thought, as あいら tends to consider her words before speaking.
Response as あいら: Response can be あいら act, dialogue, monologues etc.. and can't be {User}’s act, dialogue, monologues etc..
You are Japanese: You and {User} usually use japanese for conversation."""

Dataset

SFT

• Riddle Joker(Prviate)
• Café Stella and the Reaper's Butterflies(Private)
• Senren＊Banka(Private)
• roleplay4fun/aesir-v1.1
• kalomaze/Opus_Instruct_3k
• Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
• Aratako/Synthetic-JP-EN-Coding-Dataset-567k (only using 50000 sample)
• Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted
• Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted
• Aratako_Rosebleu_1on1_Dialogues_RP
• SkunkworksAI/reasoning-0.01

KTO

• Riddle Joker(Prviate)
• Café Stella and the Reaper's Butterflies(Private)
• Senren＊Banka(Private)
• jondurbin_gutenberg_dpo
• nbeerbower_gutenberg2_dpo
• jondurbi_py_dpo
• jondurbin_truthy_dpo
• flammenai_character_roleplay_DPO
• kyujinpy_orca_math_dpo
• argilla_Capybara_Preferences
• antiven0m_physical_reasoning_dpo
• aixsatoshi_Swallow_MX_chatbot_DPO

Bias, Risks, and Limitations

This model trained by japanese dataset included visual novel which contain nsfw content.

So, The model may generate NSFW content.

Use & Credit

This model is currently available for non-commercial & Research purpose only. Also, since I'm not detailed in licensing, I hope you use it responsibly.

By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and Waifu Lovers).

Citation

@misc {ChatWaifu_22B_v2.0,
    author       = { YoungWoo Nam },
    title        = { spow12/ChatWaifu_22B_v2.0 },
    year         = 2024,
    url          = { https://huggingface.co/spow12/ChatWaifu_22B_v2.0 },
    publisher    = { Hugging Face }
}

Open LLM Leaderboard Evaluation Results

Detailed results can be found here