KDI-Llama-3-Open-Ko-8B-Instruct

Inference sample Code

from transformers import AutoTokenizer
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("prismdata/KDI-Llama-3-Open-Ko-8B-Instruct",cache_dir="./", device_map = 'cuda')
tokenizer = AutoTokenizer.from_pretrained("prismdata/KDI-Llama-3-Open-Ko-8B-Instruct",cache_dir="./", device_map = 'cuda')

prompt_template = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.\nHuman: {prompt}\nAssistant:\n"
text = 'PMDU(prime minister’s delivery unit)가 어떤 역할을 하는 조직인가요?'
model_inputs = tokenizer(prompt_template.format(prompt=text), return_tensors='pt').to("cuda:0")

outputs = model.generate(**model_inputs, max_new_tokens=256).to("cuda:0")
output_text = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
print(output_text)

A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
Human: PMDU(prime minister’s delivery unit)가 어떤 역할을 하는 조직인가요?
Assistant:
PMDU는 총리실 산하에 있는 조직으로, 정책효과성 증대를 위한 집행과학에 관한 연구에서 총리실의 정책조정과 집행을 지원하는 역할을 합니다.