Downloads · 30 days
9
2% of all-time downloads
rsonavane/flan-t5-xl-alpaca-dolly-lora-peft
flan-t5-xl-alpaca-dolly-lora-peft is a text generation model from rsonavane. Use it when you need the model to write or continue text. It is set up for peft.
Find below some example scripts on how to use the model in transformers:
Downloads · 30 days
9
2% of all-time downloads
All-time downloads
541
Public
Repo size
4.6 GB
Likes
1
Public
Click a slice to open those files.
.bin4.6 GB · 100%
From the Hugging Face model README
Find below some example scripts on how to use the model in transformers:
import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
# Load peft config for pre-trained checkpoint etc.
peft_model_id = "rsonavane/flan-t5-xl-alpaca-dolly-lora-peft"
config = PeftConfig.from_pretrained(peft_model_id)
# load base LLM model and tokenizer
model = AutoModelForSeq2SeqLM.from_pretrained(config.base_model_name_or_path, load_in_8bit=True, device_map={"":0})
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
# Load the Lora model
model = PeftModel.from_pretrained(model, peft_model_id, device_map={"":0})
def generate_prompt(instruction: str, input_ctxt: str = "") -> str:
if input_ctxt:
return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Input:
{input_ctxt}
### Response:"""
else:
return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Response:"""
input_ctxt = ""
instruction = ""
input_text = generate_prompt(instruction, input_ctxt)
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
Intended for conversation analysis, closed qna and summarization. Trained on instructions from doll-15k, alpaca-52k and samsum dataset.