falcon-lora-imdb

🦅 Falcon-RW-1B Fine-Tuned with QLoRA on IMDB

This model is a QLoRA fine-tuned version of tiiuae/falcon-rw-1b trained on the IMDB movie review dataset for text generation.

Only the LoRA adapter weights are stored here — the base model is loaded separately from HuggingFace at runtime.

---

🧠 Model Details

---

🔧 Training Configuration

LoRA Hyperparameters

Quantization (QLoRA)

Training Arguments

---

🚀 How to Use

Installation

pip install transformers peft accelerate bitsandbytes torch

Inference (QLoRA — Recommended for low VRAM)

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel, PeftConfig
import torch

adapter_path = "Abhishek161121/falcon-rw-1b-qlora-imdb"

# Load config
peft_config = PeftConfig.from_pretrained(adapter_path)

# Load base model in 4-bit
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

base_model = AutoModelForCausalLM.from_pretrained(
    peft_config.base_model_name_or_path,
    quantization_config=bnb_config,
    trust_remote_code=True,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_path)
model.eval()

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    peft_config.base_model_name_or_path,
    trust_remote_code=True
)
tokenizer.pad_token = tokenizer.eos_token

# Generate text
prompt = "The movie was absolutely wonderful because"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=100,
        do_sample=True,
        temperature=0.8,
        top_k=50,
        top_p=0.95,
        pad_token_id=tokenizer.eos_token_id
    )

result = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(result)

Example Output

Prompt:  "The movie was absolutely wonderful because"
Output:  "The movie was absolutely wonderful because of its
          heartfelt performances and gripping storyline that
          kept me on the edge of my seat throughout..."

---

📤 How to Push This Model to HuggingFace

Run this after training to upload your adapter:

from huggingface_hub import login

login(token="YOUR_HF_TOKEN")  # Get from https://huggingface.co/settings/tokens

model.push_to_hub("YOUR_HF_USERNAME/falcon-rw-1b-qlora-imdb")
tokenizer.push_to_hub("YOUR_HF_USERNAME/falcon-rw-1b-qlora-imdb")

---

📊 Dataset

• Dataset: IMDB
• Split used: Train (500 samples subset)
• Task: Causal language modeling on movie reviews

---

⚠️ Limitations

• Trained on only 500 samples for 1 epoch — outputs may be repetitive or incoherent for long generations
• Fine-tuned for text continuation on movie review style text; not suitable for instruction following
• Base model is 1B parameters — smaller than production-grade LLMs
• May hallucinate or produce irrelevant content on out-of-domain prompts

---

📚 References

• Falcon-RW-1B
• QLoRA Paper — Dettmers et al. 2023
• PEFT Library
• BitsAndBytes
• IMDB Dataset

---

👤 Author

Fine-tuned by [Your Name] · Your HuggingFace Profile