distilgpt2-spam-detector

DistilGPT2 Spam Detector

This model fine-tunes distilgpt2 on the SMS Spam Collection dataset to classify text messages as spam or ham, framed as a text-generation task.

How it works

The model is trained on examples formatted as:

Message: <text>
Label: <ham or spam>

At inference time, feed it "Message: <your text> Label:" and let it generate a few tokens — it will produce "spam" or "ham".

Example usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("corruptedzayn/distilgpt2-spam-detector")
model = AutoModelForCausalLM.from_pretrained("corruptedzayn/distilgpt2-spam-detector")

prompt = "Message: Win a free iPhone now, click here!
Label:"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=5, pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training details

• Base model: distilgpt2
• Dataset: sms_spam (~5,500 messages, 80/20 train/test split)
• Epochs: 3
• Framed as causal language modeling, not a classification head

Limitations

This is a small demo model trained on a small, English-only, slightly dated dataset. It is not intended for production spam filtering.