fine_tuned_distilgpt2_dialogsum

Model Card for Rohit462/fine_tuned_distilgpt2_dialogsum

A DistilGPT-2 model fine-tuned using LoRA on the DialogSum dataset for English dialogue summarization. Generates concise summaries for dialogues in various topics like meetings, plans, and conversations.

Model Details

Model Description

• Developed by: Rohit Rawat
• Model type: Causal Language Model (GPT-2) with LoRA
• Language(s): English
• License: Apache-2.0
• Finetuned from: distilgpt2

Model Sources

• Repository: https://huggingface.co/Rohit462/fine_tuned_distilgpt2_dialogsum

Uses

Direct Use

Useful for generating summaries from short conversations. Example: Summary: Two friends discussed their weekend plans. Topic: Weekend planning

Downstream Use

Can be integrated into meeting tools, chatbot logs, and dialogue-based analytics.

Out-of-Scope Use

• Non-English text
• Factual or long-form summarization
• High-risk applications

Bias, Risks, and Limitations

• May reflect biases from the DialogSum dataset
• Accuracy may degrade on complex or domain-specific dialogue

Recommendations

• Human validation is recommended
• Avoid use in critical or factual applications

How to Get Started with the Model

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer, pipeline

model = AutoPeftModelForCausalLM.from_pretrained("Rohit462/fine_tuned_distilgpt2_dialogsum", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Rohit462/fine_tuned_distilgpt2_dialogsum")
tokenizer.pad_token = tokenizer.eos_token

generator = pipeline("text-generation", model=model, tokenizer=tokenizer, device=model.device)
prompt = "Summary: Two friends planned a trip. Topic: Travel discussion"
output = generator(prompt, max_new_tokens=50, do_sample=True, top_p=0.9, temperature=0.7)
print(output[0]["generated_text"])

Training Details
Training Data
Subset of the DialogSum dataset, ~1000 samples used for fine-tuning.

Training Procedure
LoRA applied on c_attn layers

Epochs: 1

Batch Size: 4

LR: 2e-5 → 1e-5

Max length: 160

FP16 precision

Platform: Google Colab T4

Evaluation
Metric: Perplexity (TBD)

Evaluation: Manual review of summary coherence and topic alignment

Environmental Impact
Hardware Type: Google Colab T4 GPU

Training Time: ~30 minutes

Carbon Emitted: < 100g CO2eq (estimated)

Technical Specifications
Architecture: DistilGPT-2 with LoRA (c_attn only)

Libraries: Hugging Face Transformers, PEFT, TRL, PyTorch

Citation
BibTeX:
Training Details
Training Data
Subset of the DialogSum dataset, ~1000 samples used for fine-tuning.

Training Procedure
LoRA applied on c_attn layers

Epochs: 1

Batch Size: 4

LR: 2e-5 → 1e-5

Max length: 160

FP16 precision

Platform: Google Colab T4

Evaluation
Metric: Perplexity (TBD)

Evaluation: Manual review of summary coherence and topic alignment

Environmental Impact
Hardware Type: Google Colab T4 GPU

Training Time: ~30 minutes

Carbon Emitted: < 100g CO2eq (estimated)

Technical Specifications
Architecture: DistilGPT-2 with LoRA (c_attn only)

Libraries: Hugging Face Transformers, PEFT, TRL, PyTorch

Citation
BibTeX:
@misc{fine_tuned_distilgpt2_dialogsum,
  author = {Rohit Rawat},
  title = {Rohit462/fine_tuned_distilgpt2_dialogsum},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Rohit462/fine_tuned_distilgpt2_dialogsum}}
}

APA:
Rohit Rawat. (2025). Rohit462/fine_tuned_distilgpt2_dialogsum. Hugging Face. https://huggingface.co/Rohit462/fine_tuned_distilgpt2_dialogsum

Model Card Contact
For questions or issues, open a discussion at:
https://huggingface.co/Rohit462/fine_tuned_distilgpt2_dialogsum/discussions