Downloads · 30 days
76
4% of all-time downloads
RichardErkhov/gangyeolkim_-_open-llama-2-ko-7b-summarization-gguf
gangyeolkim_-_open-llama-2-ko-7b-summarization-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
76
4% of all-time downloads
All-time downloads
2.1K
Public
Repo size
78.2 GB
Likes
0
Public
Click a slice to open those files.
.gguf90.6 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
open-llama-2-ko-7b-summarization - GGUF
Original model description:
AI hub에 있는 아래 요약 데이터 중 3만건을 샘플링하여 사용하였습니다.
pip3 install transformers gradio vllm
from vllm import LLM, SamplingParams
from transformers import AutoTokenizer
import gradio as gr
import os
model_path = "gangyeolkim/open-llama-2-ko-7b-summarization"
sampling_params = SamplingParams(max_tokens=1024, temperature=0.1)
tokenizer = AutoTokenizer.from_pretrained(model_path)
llm = LLM(model=model_path, tokenizer=model_path, tensor_parallel_size=1)
def gen(text, history):
text = [
"### 원문:",
f"{text}\n",
"### 요약:\n",
]
prompts = "\n".join(text)
outputs = llm.generate(prompts, sampling_params)
for output in outputs:
generated_text = output.outputs[0].text
return generated_text
demo = gr.ChatInterface(gen)
demo.launch(share=True)