Bel_qwen3.8-4B

Qwen3.8-4B-Distill Belarusian LoRA

LoRA-адаптер для белорусского языка, обученный поверх empero-ai/Qwen3.8-4B-Distill.

Детали обучения

Использование

Вариант 1: С 4-bit квантованием (рекомендуется, работает на GPU от 8 ГБ)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

model_id = "empero-ai/Qwen3.8-4B-Distill"
adapter_id = "Aleton/Bel_qwen3.8-4B"

# 4-bit квантование для экономии VRAM
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)

base = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map={"": 0},
)
model = PeftModel.from_pretrained(base, adapter_id)
tokenizer = AutoTokenizer.from_pretrained(adapter_id)

messages = [{"role": "user", "content": "Прывітанне! Як справы?"}]
inputs = tokenizer.apply_chat_template(
    messages, 
    add_generation_prompt=True, 
    return_tensors="pt",
    return_dict=True
).to(model.device)

out = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7)

input_length = inputs.input_ids.shape[-1]
response = tokenizer.decode(out[0][input_length:], skip_special_tokens=True)
print(response)

Требования:

pip install torch transformers peft bitsandbytes accelerate

---

Вариант 2: Без квантования (требуется GPU с 16+ ГБ VRAM)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model_id = "empero-ai/Qwen3.8-4B-Distill"
adapter_id = "Aleton/Bel_qwen3.8-4B"

base = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map={"": 0},
)
model = PeftModel.from_pretrained(base, adapter_id)
tokenizer = AutoTokenizer.from_pretrained(adapter_id)

messages = [{"role": "user", "content": "Прывітанне! Як справы?"}]
inputs = tokenizer.apply_chat_template(
    messages, 
    add_generation_prompt=True, 
    return_tensors="pt",
    return_dict=True
).to(model.device)

out = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7)

input_length = inputs.input_ids.shape[-1]
response = tokenizer.decode(out[0][input_length:], skip_special_tokens=True)
print(response)

---