kanelsnegl-v0.2-GGUF

Model Card for kanelsnegl-v0.2-GGUF

!!! This model is built for fun and learning and needs much more finetuning !!! For now only Q4_K.

<img src="https://huggingface.co/RJuro/kanelsnegl-v0.2-GGUF/resolve/main/banner_ks_s.jpg" alt="Kanelsnegl Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>

Model Description

Base model: Zephyr-7b-alpha finetuned on DDSC/partial-danish-gigaword-no-twitter. The training involved a maximum length of 512. QLora completion finetuning of all linear layers was also implemented. This model is mostly fun tinkering for personal learning purpose. A heavily instruction-tuned model was used in the hope of transferring some of its behaviors into the Danish finetune.

The model often produces somewhat fun halucinations and instruction following is not really happening that well, but there may be ways to fine-tune it in a supervised manner for more controlled behavior.

Works with Ollama but not too well

Usage with CTransformers

from ctransformers import AutoModelForCausalLM

# Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
llm = AutoModelForCausalLM.from_pretrained('RJuro/kanelsnegl-v0.2-GGUF', 
                                           model_file="kanelsnegl-v0.2-Q4_K.gguf", 
                                           model_type="mistral", 
                                           gpu_layers=50
                                           )

print(llm("""<|system|> Du er en god og sød assistent </s>

<|user|>Hvad kan du sige om Aalborg?</s>
""", 
          top_k=50, 
          top_p=0.95, 
          temperature=0.2, 
          repetition_penalty=1.2, 
          reset=True,
          max_new_tokens=256))

Returns 😂

<|assistant|>Aalborg er et charmerende byområde i det nordlige Jylland med mange attraktive steder at besøge.
Byen har en rigtig god atmosfære og er kendt for sin livelige natliv, men også som en velfungerende hovedstad for Nordjyllands region.
Der findes mange gode restauranter, cafeer og butikker i byen, samt et rigtig godt teater og koncertscene.
Aalborg er også kendt for sin historie som en industriby, hvilket stadig kan ses i de gamle fabrikker og værksteder rundtomkring i byen.
Alt i alt er Aalborg et godt sted at besøge, både for turister og lokale indbyggere. </s>