Downloads · 30 days
4
7% of all-time downloads
sosoai/hansoldeco-beomi-llama3-8b-ko-v0.2-chatvector
hansoldeco-beomi-llama3-8b-ko-v0.2-chatvector is a text generation model from sosoai. Use it when you need the model to write or continue text. It is set up for transformers.
Downloads · 30 days
4
7% of all-time downloads
All-time downloads
61
Public
Parameters
8B
16.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16.1 GB · 100%
From the Hugging Face model README
used chat vector extraction method
base model = beomi/Llama-3-Open-Ko-8B v2 base model = sosoai/hansoldeco-beomi-llama3-8b-ko-v0.1 (hansoldeco domain own dataset)
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
import torch
tokenizer = AutoTokenizer.from_pretrained("sosoai/hansoldeco-beomi-llama3-8b-ko-v0.1")
model_name = "sosoai/hansoldeco-beomi-llama3-8b-ko-v0.2-chatvector"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto",
)
conversation = [
{'role': 'user', 'content': "안녕하세요! 너는 누구세요?"},
{'role': 'assistant', 'content': "저는 한솔데코 도배, 벽지 그리고 마감자재 하자 전문 챗봇입니다. 무엇을 도와드릴까요? 이와 관련된 대한 모든 질문을 해주세요."},
{'role': 'user', 'content': "안녕하세요! 당신의 이름을 알려주세요. 벽지하자는 어떻게 해결해요?"}
]
prompt = tokenizer.apply_chat_template(conversation, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, use_cache=True, max_length=256)
output_text = tokenizer.decode(outputs[0])
print(output_text)