Downloads · 30 days
7
19% of all-time downloads
RichardErkhov/piotr-ai_-_polanka-qwen2-3b-v0.3-chat-4bits
piotr-ai_-_polanka-qwen2-3b-v0.3-chat-4bits is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
7
19% of all-time downloads
All-time downloads
37
Public
Parameters
3.5B
3.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.5 GB · 100%
How the weights are stored.
U82.3B · 67%
From the Hugging Face model README
Quantization made by Richard Erkhov.
polanka-qwen2-3b-v0.3-chat - bnb 4bits
license: apache-2.0 language:
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
import os
device = "cuda"
device_map = {
"lm_head": device,
"model": device
}
model_name = "piotr-ai/polanka-qwen2-3b-v0.3-chat"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map=device_map)
prompt = "Co to jest psychologia?"
messages = [
{"role": "system", "content": "Jesteś przydatnym asystentem."},
{"role": "assistant", "content": "Witam, jak mogę pomóc?"},
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(messages, tokenize=False)
model_inputs = tokenizer([text], return_tensors="pt").to(device)
generated_ids = model.generate(
model_inputs.input_ids,
max_new_tokens=256,
num_return_sequences=1,
eos_token_id=tokenizer.convert_tokens_to_ids('<|im_end|>'),
pad_token_id=tokenizer.eos_token_id
)
print(tokenizer.decode(generated_ids[0]))
<|im_start|>system
Jesteś przydatnym asystentem.<|im_end|>
<|im_start|>assistant
Witam, jak mogę pomóc?<|im_end|>
<|im_start|>user
Co to jest psychologia?<|im_end|>
<|im_start|>assistant
Psychologia to nauka badająca zachowanie i procesy umysłowe. Jest to gałąź psychologii, która koncentruje się na zrozumieniu i wyjaśnianiu zachowań i procesów umysłowych.<|im_end|>