distilbert-ag-news-lora-merged

DistilBERT AG News (LoRA fine-tuned)

A LoRA fine-tuned DistilBERT model for AG News classification (Apache 2.0 licensed).

This model is a fine-tuned version of DistilBERT base uncased using LoRA (Low-Rank Adaptation) for parameter-efficient fine-tuning.
It has been trained on the AG News dataset to perform news headline classification into the following categories:

• World
• Sports
• Business
• Sci/Tech

---

Model Details

• Base model: distilbert-base-uncased
• Fine-tuning method: LoRA
• Dataset: AG News
• Task: Text classification (4 classes)
• Author: Gemma Lara Savill

---

Usage

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="gemmalarasav/distilbert-ag-news-lora-merged"
)

text = "OpenAI releases new GPT model"
result = classifier(text)
print(result)

Example output:

[{'label': 'Sci/Tech', 'score': 0.987}]

---

Performance

• Fine-tuned accuracy: ~90% on AG News test split
• Lightweight (LoRA) → smaller footprint and faster adaptation compared to full fine-tuning

---

License and Attribution

This model is a fine-tuned version of DistilBERT,
which is licensed under the Apache License 2.0.

This fine-tuned model is released under the same license: Apache 2.0.
See the included LICENSE file for details.

---

Limitations and Biases

• The model inherits potential biases from DistilBERT and the AG News dataset.
• It may misclassify edge cases or headlines outside the AG News domain.
• Not suitable for sensitive decision-making without human oversight.