Downloads · 30 days
27
30% of all-time downloads
DistillLens-gpt2-base
manishdhakal/DistillLens-gpt2-base
DistillLens-gpt2-base is a text generation model from manishdhakal. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
All-time downloads
91
Public
Parameters
124M
249 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors249 MB · 99%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Model type
- gpt2
- Access
- Public
- Created
- Feb 28, 2026
- Updated
- Feb 28, 2026
- SHA
- 2841907d
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
DistillLens-gpt2-base
DistillLens-gpt2-base is a gpt2-base (120M) model distilled from gpt2-xlarge (1.5B) on databricks-dolly-15k.
Method
<p align='left'> <img src="architecture.png" width="1000"> </p>Note: DistillLens requires a GPT2-Base for initilization to perform the distillation.
Citation
@article{dhakal2026distilllens,
title={DistillLens: Symmetric Knowledge Distillation Through Logit Lens},
author={Dhakal, Manish and Jinadu, Uthman and Budathoki, Anjila and Sunderraman, Rajshekhar and Ding, Yi},
journal={arXiv preprint arXiv:2602.13567},
year={2026}
}
- Task
- Text Generation
- Type
- gpt2
- License
- apache-2.0
- Languages
- en
- Created
- Feb 28, 2026
- Updated
- Feb 28, 2026