Downloads · 30 days
26
1% of all-time downloads
akdeniz27/llama-2-7b-hf-qlora-dolly15k-turkish
llama-2-7b-hf-qlora-dolly15k-turkish is a text generation model from akdeniz27. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as other.
The following bitsandbytes quantization config was used during training: - loadin8bit: False - loadin4bit: True - llmint8threshold: 6.0 - llmint8skipmodules: None - llmint8enablefp32cpuoffload: False - llmint8hasfp16w…
Downloads · 30 days
26
1% of all-time downloads
All-time downloads
2.4K
Public
Repo size
269 MB
Likes
8
Public
Click a slice to open those files.
.bin134 MB · 100%
From the Hugging Face model README
The following bitsandbytes quantization config was used during training:
!pip install transformers peft accelerate bitsandbytes trl safetensors
from huggingface_hub import notebook_login
notebook_login()
import torch
from peft import AutoPeftModelForCausalLM, get_peft_config, PeftModel, PeftConfig, get_peft_model, LoraConfig, TaskType
from transformers import AutoTokenizer
peft_model_id = "akdeniz27/llama-2-7b-hf-qlora-dolly15k-turkish"
config = PeftConfig.from_pretrained(peft_model_id)
# load base LLM model and tokenizer
model = AutoPeftModelForCausalLM.from_pretrained(
peft_model_id,
low_cpu_mem_usage=True,
torch_dtype=torch.float16,
load_in_4bit=True,
)
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
prompt = "..."
input_ids = tokenizer(prompt, return_tensors="pt", truncation=True).input_ids.cuda()
outputs = model.generate(input_ids=input_ids, max_new_tokens=100, do_sample=True, top_p=0.9,temperature=0.9)