Downloads · 30 days
5
4% of all-time downloads
sartajbhuvaji/llama-2-7b-resonate-v1
llama-2-7b-resonate-v1 is a text generation model from sartajbhuvaji. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
Downloads · 30 days
5
4% of all-time downloads
All-time downloads
120
Public
Parameters
6.7B
13.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors13.6 GB · 100%
From the Hugging Face model README
from peft import AutoPeftModelForCausalLM, LoraConfig, PeftModel
from transformers import AutoTokenizer, pipeline
import torch
base_model_name = "NousResearch/Llama-2-7b-chat-hf"
qlora_model_adapter = "sartajbhuvaji/llama-2-7b-resonate-v1"
device_map = {"": 0}
base_model = AutoModelForCausalLM.from_pretrained(
base_model_name,
low_cpu_mem_usage=True,
return_dict=True,
torch_dtype=torch.float16,
device_map=device_map,
)
model = PeftModel.from_pretrained(base_model, qlora_model_adapter)
model = model.merge_and_unload()
tokenizer = AutoTokenizer.from_pretrained(base_model_name, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"
prompt = "What is a large language model?"
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=2000)
result = pipe(f"<s>[INST] {prompt} [/INST]")
print(result[0]['generated_text'])