text-summarizer

Text Summarizer – Transformer Model

This model is a fine-tuned sequence-to-sequence Transformer for abstractive text summarization. It generates concise summaries from long input paragraphs using beam search decoding.

Model Details Model Description

This model is fine-tuned for abstractive text summarization using a Transformer-based sequence-to-sequence architecture. It takes long-form textual input and generates a coherent summary while preserving key information.

Developed by: Sarvesh Chhabra

Model type: Sequence-to-Sequence Transformer

Language(s): English

License: Apache 2.0

Finetuned from model: T5-base

Model Sources

Repository: https://huggingface.co/sarvesh77/text-summarizer

Demo API: FastAPI-based deployment (local/production)

Uses Direct Use

This model can be used for:

News summarization

Academic paragraph summarization

Document condensation

Educational content simplification

Downstream Use

Integrated into web applications (FastAPI)

Deployed as REST API

Used in research or NLP pipelines

Integrated into document processing systems

Out-of-Scope Use

Legal or medical critical summarization without verification

High-stakes decision systems

Real-time large-scale enterprise deployment without optimization

Bias, Risks, and Limitations

This model:

May generate incomplete or slightly inaccurate summaries

May omit important details in long documents

Is limited to English language input

Reflects biases present in training data

Recommendations

Users should:

Verify generated summaries before critical use

Avoid use in high-risk domains without human review

Consider domain-specific fine-tuning for better performance

How to Get Started from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "sarvesh77/text-summarizer"

tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

text = "Artificial intelligence is transforming industries by automating tasks and improving decision making."

inputs = tokenizer("summarize: " + text, return_tensors="pt", truncation=True) summary_ids = model.generate(**inputs, max_length=60, num_beams=4)

print(tokenizer.decode(summary_ids[0], skip_special_tokens=True)) Training Details Training Data

Fine-tuned on the CNN/DailyMail dataset

Dataset includes news articles paired with human-written highlights

Preprocessing

Added prefix: "summarize: " to inputs

Truncated input to 512 tokens

Truncated target summaries to 128 tokens

Training Hyperparameters

Learning rate: 5e-5

Batch size: 2

Epochs: 1

Weight decay: 0.01

Beam size: 4

Training regime: fp32

Evaluation Metrics

ROUGE-1

ROUGE-2

ROUGE-L

Results (Sample Evaluation) Metric Score ROUGE-1 ~0.40 ROUGE-2 ~0.18 ROUGE-L ~0.36

(Scores may vary depending on evaluation subset.)

Model Architecture and Objective

Architecture: T5 (Text-to-Text Transfer Transformer)

Objective: Conditional generation (sequence-to-sequence)

Decoding Strategy: Beam search

Compute Infrastructure Hardware

Local GPU / CPU training environment

Software

Python 3.10

PyTorch

Hugging Face Transformers

Datasets & Evaluate

Environmental Impact

Hardware Type: Local GPU

Hours Used: ~1–2 hours

Cloud Provider: Local machine

Carbon Emitted: Minimal (small-scale fine-tuning)

Citation

If you use this model, please cite:

APA:

Chhabra, S. (2025). Transformer-based Text Summarization Model. Hugging Face Hub.