Downloads · 30 days
18
2% of all-time downloads
sh2orc/llama-3-Korean-Bllossom-70B-AWQ
llama-3-Korean-Bllossom-70B-AWQ is a text generation model from sh2orc. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.
- AWQ Quantization from Bllossom, Meta llama-3-70b full-finetuned by Korean dataset - vLLM, Paged Attention
Downloads · 30 days
18
2% of all-time downloads
All-time downloads
1K
Public
Parameters
70.8B
40.3 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors40.3 GB · 100%
How the weights are stored.
I3268.5B · 97%
From the Hugging Face model README
instruction = '파이썬 merge sort 코드를 제시해줘'
messages = [
{
"role": "system",
"content": "당신은 훌륭한 AI 비서입니다. 짧은 답변을 제시하고, 다음으로 상세 설명을 해주세요. You are a great AI assistant. Give a short answer, then elaborate. "
},
{
"role": "user",
"content": instruction
},
]
prompt_message = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
eos_token_id = [tokenizer.eos_token_id, tokenizer.convert_tokens_to_ids("<|eot_id|>")]
outputs = llm.generate(prompt_message, SamplingParams(stop_token_ids=eos_token_id, temperature=0.8, top_p=0.95,max_tokens=512))
for output in outputs:
propt = output.prompt
generated_text = output.outputs[0].text
print(generated_text)