anime-personality-chat-gguf

Overview

This repository contains a GGUF format model, which is a quantized version of Mistral-7B-Instruct natively merged with the maomao88/anime-waifu-mistral-lora adapter. The base model was fine-tuned using Low-Rank Adaptation (LoRA) to adapt Mistral's conversational architecture into a highly stylized, engaging, and expressive anime-themed persona, and has been fully baked into the model weights prior to quantization.

Run model with llama_cpp

1. Install dependency

pip install llama-cpp-python

1. Load the model

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="maomao88/anime-personality-chat-gguf",
    filename="mistral-merged-Q4_K_M.gguf",
    n_ctx=2048,       # Set context window token ceiling limit
    n_gpu_layers=-1   # Set to -1 to push ALL layers straight into GPU VRAM (0 for pure CPU)
)

1. Run the model

def chat_with_personality(trait, user_input, max_new_tokens=100, temperature=0.8):
    """
    Generate a response from the fine-tuned model using a given personality trait.
    """
    messages = [
        {
            "role": "system",
            "content": f"You are an anime character with the following personality: {trait}."
        },
        {
            "role": "user",
            "content": user_input
        }
    ]

    # Map the arguments properly to your function parameters
    response_dict = llm.create_chat_completion(
        messages=messages,
        temperature=temperature,
        max_tokens=max_new_tokens
    )

    # Extract the text string from the OpenAI-style dictionary payload
    assistant_reply = response_dict['choices'][0]['message']['content'].strip()

    return assistant_reply

chat_with_personality("tsundere", "What do you think of the moon?")
# Response Example: I mean, it’s not like the moon is super romantic or anything, right?

The anime characters supported are as follows:

tsundere (傲娇)
yandere (病娇)
himedere (公主娇)
genki (元气)
moe (萌系)
bakadere (笨蛋)