llama-3-8b-customer-support-qlora

Llama-3-8B QLoRA — Customer Support

A LoRA adapter that fine-tunes Meta-Llama-3-8B-Instruct into a customer-support assistant, trained with QLoRA on the Bitext customer-support dataset (~26.9K examples across 27 support intents: cancellations, refunds, shipping, account management, payments, and more).

Held-out perplexity improved 79% over the base model (8.87 → 1.87), while training only 0.17% of the model's 8 billion parameters.

Full write-up, methodology, and a live base-vs-fine-tuned Q&A comparison: GitHub repo.

Method

Results

Qualitatively, the base model tends to fabricate fake multi-turn dialogue or occasionally hallucinate an unrelated domain (e.g. answering a refund question with IRS tax-refund instructions), while the fine-tuned adapter stays on-topic, concise, and in-character as a support agent — including on questions phrased differently from the training data's exact wording.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_model_id = "meta-llama/Meta-Llama-3-8B-Instruct"
adapter_id = "AmmarNasir/llama-3-8b-customer-support-qlora"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
tokenizer.pad_token = tokenizer.eos_token

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

prompt = "### Human: I want to cancel my order### Assistant:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=128, do_sample=True, temperature=0.7, pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Note the prompt format: ### Human: <message>### Assistant: — this matches the format used during training.

Training details

• Trained on the full dataset (24,184 train / 2,688 held-out eval examples, a 90/10 split), with automatic early stopping (EarlyStoppingCallback, patience of 5 evaluation checks on held-out eval_loss) rather than a fixed epoch count, since this dataset's templated response style converges quickly and overfitting is a bigger risk than underfitting.
• This checkpoint (eval_loss 0.6384 on the training-time held-out split) was training's best checkpoint at the point training was interrupted by an unrelated infrastructure event (a WSL2 VM restart) — not by convergence. eval_loss was still improving at every checkpoint up to this point.

License

This adapter is derived from Meta-Llama-3-8B-Instruct and is subject to the Meta Llama 3 Community License. The training dataset is licensed under CDLA-Sharing-1.0.