falcon-ecommerce-qa

Falcon-1B Fine-tuned for E-Commerce Q&A

Model Description

This model is a fine-tuned version of tiiuae/falcon-rw-1b using QLoRA (4-bit quantization) on an e-commerce product Q&A dataset. It generates helpful product review responses to customer queries.

Training Details

• Base model: tiiuae/falcon-rw-1b
• Technique: QLoRA with 4-bit quantization (BitsAndBytes NF4)
• LoRA rank: r=8, alpha=32
• Trainable parameters: 1,572,864 (0.12% of total)
• GPU: Tesla T4 (15GB) — memory usage: ~2.6GB
• Epochs: 2
• Final training loss: ~0.42
• Dataset: ecommerce-qa-dataset

Key Results

• 50%+ GPU memory reduction via 4-bit quantization
• Loss converged from 7.8 → 0.42 over 2 epochs
• Only 0.12% of parameters trained (LoRA adapters)

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-rw-1b")
model = PeftModel.from_pretrained(base, "Srushti-Thakur013/falcon-ecommerce-qa")
tokenizer = AutoTokenizer.from_pretrained("Srushti-Thakur013/falcon-ecommerce-qa")

Intended Use

E-commerce product Q&A, customer support response generation.