llama2-7f-moviechatbot-ggml-q4

Llama2 Movie Character Finetuned 7F Quantized

Quantized by llama.cpp (https://github.com/ggerganov/llama.cpp) Either use by llama.cpp or llama cpp python wrappers

with ctransformers:

#pip install ctransformers
from ctransformers import AutoModelForCausalLM

llm = AutoModelForCausalLM.from_pretrained("llama2-7f-fp16-ggml-q4.bin", 
                                           model_type='llama',
                                           gpu_layers=100,  # you can use less (like 20) if you have less gpu ram
                                           max_new_tokens=50, 
                                           stop=["###","##"],
                                           threads=4)  # 4 cpu threads to limit cpu

with llama-cpp-python:

# !CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python

model_path="llama2-7f-fp16-ggml-q4.bin"
llm = Llama(model_path=model_path,n_gpu_layers=100)
output = llm("### Context: talks friendly### History: ### Morpheus: I challenge you to battle of words!### Gerald:", max_tokens=32, stop=["#","sierpeda"], echo=True)

Prompt format

prompt = f"Context: {overall_context}### History: {previous_dialogue} ### {character_1}: {question}### {character_2}:"

Some Fun dialogues

Geralt of Witcher vs Ig-11 of Mandalorian

context= "on a cruise ship going to Mars from Earth"

#Gerald(Geralt Witcher, do not know why I named him incorrect) asks Ig-11 (Mandalorian) initial_text = What do you think about AI?

Gerald : What do you think about AI?
Ig-11 : I've worked with them for more than half of my operational cycle.
Gerald :  So, how long has the ship been here? 
Ig-11 : 765 years (33 days) since last port.
Gerald :  So this is the fourth time since I joined the ship that we've had a murderer on board?
Ig-11 : My master does not believe in coincidences... He believes everything happens for a reason. The universe has a plan, and it includes me being here at the same time as you. So, what do you think about AI?
Gerald :  I'm not sure yet... I mean, they work on Earth, and maybe beyond.
Ig-11 : You see? It is all so simple! The humans are going to Mars; we are bringing them with us.

Batman vs Tony Stark

context= "tries to save the world"

initial_text = "I am richer than you!"

Batman : I am richer than you!
Tony Stark : My car goes 100mph!
Batman :  My car can go at least that fast, and it's not even suped up. everybody in the car. It didn't even have a license plate! It was so loud I couldn't hear myself think. It felt like we were gonna get killed any second... 
Tony Stark : You need to chill out, Bruce. You can be such a bitch sometimes.~Tony Stark
Batman :  You don't get it do you? It's not about justice. It's about power.~Bruce Wayne
Tony Stark : You know the one thing that really gets under my skin? When someone shoots himself in the foot so bad, he ends up putting on a hobble~Tony Stark
Batman :  "Let me tell you something. Something I've learned to my great cost..."~Batman
Tony Stark : "You make it sound like a disease."

Darth Vader vs James T. Kirk

Vader : I challenge you to battle of words!
Kirk : What are your terms?
Vader :  The Emperor's terms!
Kirk : I challenge you to the ultimate game, a verbal battle of wit and wile. surely you know that no mere mortal can stand against my brilliance.
Vader :  I accept your challenge. The terms I propose is, the winner shall have the pleasure of killing the loser!
Kirk : Deal! But you will be disappointed. Though my words are powerful and sharp, they lack the strength to kill.
Vader :  You speak truthfully. My vocabulary lacks power. It is but a fraction of what it was 10 years ago at the height of my powers.
Kirk : I've never heard you speak like this before.