Downloads · 30 days
0
agnews-distilbert-lora
starkdv123/agnews-distilbert-lora
agnews-distilbert-lora is a text classification model from starkdv123. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
This repository contains LoRA adapter weights trained on AG News for DistilBERT.
Access
Public
Updated Sep 22, 2025
Repo size
3.6 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.6 MB · 79%
At a glance
- Task
- Text Classification
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Sep 22, 2025
- Updated
- Sep 22, 2025
- SHA
- 472888be
Try a prompt
Datasets
Model card
From the Hugging Face model README
DistilBERT for AG News — LoRA Adapter (PEFT)
This repository contains LoRA adapter weights trained on AG News for DistilBERT.
Results (reference)
| Model | Test Accuracy | Macro F1 |
|---|---|---|
| LoRA (merged) | 0.9400 | 0.9400 |
Confusion Matrix (Merged, Test)
World Sports Busines Sci/Tec
World 1791 14 49 46
Sports 12 1874 9 5
Busines 42 8 1718 132
Sci/Tec 30 9 100 1761
Usage (attach to base)
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline
from peft import PeftModel
base = "distilbert-base-uncased"
adapter = "starkdv123/agnews-distilbert-lora"
tok = AutoTokenizer.from_pretrained(base)
base_model = AutoModelForSequenceClassification.from_pretrained(base, num_labels=4)
model = PeftModel.from_pretrained(base_model, adapter)
clf = pipeline("text-classification", model=model, tokenizer=tok, truncation=True)
clf(["New AI chip announced for smartphones."])
Training (summary)
- LoRA: r=8, α=16, dropout=0.05, LR=2e-4, epochs=2, batch=16, max_len=256
- Targets: [q_lin, k_lin, v_lin, out_lin]
Author
Karan D Vasa — https://huggingface.co/starkdv123
- Task
- Text Classification
- Library
- transformers
- License
- apache-2.0
- Languages
- en
- Created
- Sep 22, 2025
- Updated
- Sep 22, 2025