Downloads · 30 days
238
7% of all-time downloads
RichardErkhov/AIdenU_-_Mistral-7B-v0.2-ko-Y24_v2.0-gguf
AIdenU_-_Mistral-7B-v0.2-ko-Y24_v2.0-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
238
7% of all-time downloads
All-time downloads
3.4K
Public
Repo size
82.2 GB
Likes
0
Public
Click a slice to open those files.
.gguf95.2 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
Mistral-7B-v0.2-ko-Y24_v2.0 - GGUF
license: apache-2.0 language:
from transforemrs import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("AIdenU/Mistral-7B-v0.2-ko-Y24_v2.0", device_map="auto", torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained("AIdenU/Mistral-7B-v0.2-ko-Y24_v2.0", use_fast=True)
prompt = [
{'role': 'system', 'content': '당신은 지시를 매우 잘 따르는 인공지능 비서입니다.'},
{'role': 'user', 'content': '지렁이도 밟으면 꿈틀하나요?'}
]
outputs = model.generate(
**tokenizer(
tokenizer.apply_chat_template(prompt, tokenize=False, add_generation_prompt=True),
return_tensors='pt'
).to('cuda'),
max_new_tokens=256,
temperature=0.2,
top_p=1,
do_sample=True
)
print(tokenizer.decode(outputs[0]))