Llama-3-8B-TKK-Elite-V1.0-GGUF

QuantFactory/Llama-3-8B-TKK-Elite-V1.0-GGUF

This is quantized version of tarikkaankoc7/Llama-3-8B-TKK-Elite-V1.0 created using llama.cpp

Model Description

<h1 style="text-align: center;">Llama-3-TKK-8B-Elite-V1.0 </h1> <p style="text-align: center;"> Llama-3-TKK-8B-Elite-V1.0, a generative model built upon the LLaMA 8B architecture, represents my individual undergraduate graduation project. Developed during my studies in Software Engineering at Malatya Turgut Özal University, this project stands as a culmination of my academic endeavors. I extend my sincere appreciation to Assoc. Prof. Dr. Harun Bingöl, who served as both my department chair and thesis advisor. His invaluable guidance, unwavering support, and mentorship have significantly shaped my educational and research experiences. I am deeply grateful for his continuous encouragement, insightful feedback, and unwavering dedication. Thank you, Dr. Bingöl... </p>

<h2>Model Details</h2> <p> Training took 133 hours and 59 minutes for a total of 37,420 steps and was conducted on 8 Tesla V100 GPUs. </p> <ul> <li><strong>Base Model:</strong> LLaMA 8B based LLM</li> <li><strong>Model Developers:</strong> Tarık Kaan Koç</li> <li><strong>Thesis Advisor:</strong> Assoc. Prof. Dr. Harun Bingöl</li> <li><strong>Input:</strong> Text only</li> <li><strong>Output:</strong> Text only</li> <li><strong>Training Dataset:</strong> Cleaned Turkish raw data with 1 million raw instruction Turkish data, private</li> <li><strong>Training Method:</strong> Fine-tuning with LORA</li> </ul> <h2>LORA Fine-Tuning Configuration</h2>

<ul> <li><strong>lora_alpha:</strong> 16</li> <li><strong>lora_dropout:</strong> 0.1</li> <li><strong>r:</strong> 64</li> <li><strong>bias:</strong> none</li> <li><strong>task_type:</strong> CAUSAL_LM</li> </ul>

Example Usage:

from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer, pipeline
import torch

model_id = "tarikkaankoc7/TKK-LLaMA3-8B-Elite-V1.0"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True
)

streamer = TextStreamer(tokenizer)

text_generation_pipeline = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    model_kwargs={"torch_dtype": torch.bfloat16},
    streamer=streamer
)

messages = [
    {"role": "system", "content": "Sen yardımsever bir yapay zeka asistanısın ve kullanıcıların verdiği talimatlara doğrultusunda en iyi cevabı üretmeye çalışıyorsun."},
    {"role": "user", "content": "Leonardo da Vinci'nin en ünlü tablosu hangisidir?"}
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

terminators = [
    tokenizer.eos_token_id
]

outputs = text_generation_pipeline(
    prompt,
    max_new_tokens=2048,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.95
)

print(outputs[0]["generated_text"])

Output:

Leonardo da Vinci'nin en ünlü tablosu Mona Lisa'dır.