Downloads · 30 days
55
12% of all-time downloads
caretech-owl/leo-hessionai-7B-quotes-gguf
leo-hessionai-7B-quotes-gguf is a text generation model from caretech-owl. Use it when you need the model to write or continue text. The card lists the license as llama2.
This model is trained to generate german quotes for a given author. The full model can be tested at spaces/caretech-owl/quote-generator-de, here we provide a full model with a 8 bit quantization.
Downloads · 30 days
55
12% of all-time downloads
All-time downloads
473
Public
Repo size
7.2 GB
Likes
0
Public
Click a slice to open those files.
.gguf7.2 GB · 100%
From the Hugging Face model README
This model is trained to generate german quotes for a given author. The full model can be tested at spaces/caretech-owl/quote-generator-de, here we provide a full model with a 8 bit quantization.
This fine-tuned model has been trained on the caretech-owl/wikiquote-de-quotes dataset. The model was trained on a prompt like this
prompt_format = "<|im_start|>system\
Dies ist eine Unterhaltung zwischen einem\
intelligenten, hilfsbereitem KI-Assistenten und einem Nutzer.
Der Assistent gibt Antworten in Form von Zitaten.<|im_end|>\n\
<|im_start|>user\
Zitiere {author}<|im_end|>\n<\
|im_start|>assistant\n{quote}<|im_end|>\n"
Where author is itended to be provided by the user, the quote is of format quote + " - " + author.
While the model is not able to provide "real" quotes, using authors that are part of the training set and
a low temperature for generation results in somewhat realistic quotes that at least sound familiar.
import torch
from ctransformers import AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained(
"caretech-owl/leo-hessionai-7B-quotes-gguf", model_type="llama")
system_prompt = """Dies ist eine Unterhaltung zwischen \
einem intelligenten, hilfsbereitem \
KI-Assistenten und einem Nutzer.
Der Assistent gibt Antworten in Form von Zitaten."""
prompt_format = "<|im_start|>system\n{system_prompt}\
<|im_end|>\n<|im_start|>user\nZitiere {prompt}\
<|im_end|>\n<|im_start|>assistant\n"
def get_quote(author:str, max_new_tokens:int=200):
query = prompt_format.format(system_prompt=system_prompt, prompt= author)
output = base_model(query, stop='<|im_end|>', max_new_tokens=max_new_tokens, temperature=0.2, top_k=10)
print(output)
get_quote("Heinrich Heine")
The following bitsandbytes quantization config was used during training: