falcon-7b-uae-qapairs-67

PEFT Model Fine-tuned on UAE QA Pairs

This repository contains a fine-tuned model based on the PEFT framework for question answering tasks. The model has been trained on a dataset of question and answer pairs related to the UAE.

Installation

Before using the model, make sure to install the necessary packages:

pip install transformers
pip install torch torchvision
pip install peft

Usage

The model can be used for generating responses to prompts. Here is an example:

from peft import PeftModel, PeftConfig
from transformers import  AutoModelForCausalLM, AutoTokenizer
import torch

peft_model_id = "insomeniaT/falcon-7b-uae-qapairs-67"

config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, trust_remote_code=True)
model = PeftModel.from_pretrained(model, peft_model_id)

tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-7b")
tokenizer.pad_token = tokenizer.eos_token

text = "### Human: What is the minimum requirement for the UAE's GCC residency?? ### Assistant: "
device = "cuda:0"

inputs = tokenizer(text, return_tensors="pt")
inputs.to(device)
model.to(device)

outputs = model.generate(input_ids=inputs["input_ids"], attention_mask=inputs["attention_mask"], max_new_tokens=300, pad_token_id=tokenizer.eos_token_id)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)