Downloads · 30 days
0
STRLite
balaboom123/STRLite
STRLite is a image-to-text model from balaboom123. Use it when you need a caption or text from an image. It is set up for pytorch. The card lists the license as mit.
STR-Lite is an ultra-lightweight scene text recognition model that combines Masked Autoencoder (MAE) pretraining with an autoregressive decoder for text generation. With only 6M parameters, it achieves competitive acc…
Access
Public
Updated Apr 22, 2026
Repo size
139 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pth139 MB · 100%
At a glance
- Task
- Image-to-Text
- Library
- pytorch
- License
- mit
- Access
- Public
- Created
- Apr 22, 2026
- Updated
- Apr 22, 2026
- SHA
- f937a4e1
Model card
From the Hugging Face model README
STR-Lite
STR-Lite is an ultra-lightweight scene text recognition model that combines Masked Autoencoder (MAE) pretraining with an autoregressive decoder for text generation. With only 6M parameters, it achieves competitive accuracy while remaining highly efficient for real-world deployment.
- GitHub: balaboom123/STR-Lite
- Author: Kuanwei Chen
- License: MIT
Model Architecture
| Component | Details |
|---|---|
| Backbone | ViT-Tiny (embed=192, depth=12, heads=12) |
| Decoder | 1-layer autoregressive transformer (embed=192, heads=12) |
| Input size | 32 × 128 (H × W) |
| Patch size | 4 × 8 |
| Parameters | ~6M |
| Precision | bfloat16 |
Training
Stage 1 — MAE Pretraining
- Dataset: U14M-Unlabeled
- Epochs: 40
Stage 2 — Fine-tuning
- Dataset: U14M-L-Filtered
- Epochs: 20, Batch: 256, LR: 1e-3, Weight decay: 0.01
Checkpoints
| Model | Description | Epochs | Acc | Download |
|---|---|---|---|---|
| MAE ViT-Tiny | Pretrained encoder only | 40 | — | pretrain/checkpoint-last.pth |
| STRLite | Full fine-tuned model | 20 | 93.82% | finetune/checkpoint-best.pth |
Results
Common STR Benchmarks
| Subset | w/ pretrain | w/o pretrain |
|---|---|---|
| CUTE80 | 95.83 | 94.79 |
| IC13 | 96.85 | 96.50 |
| IC15 | 86.80 | 86.25 |
| IIIT5k | 96.97 | 96.47 |
| SVT | 95.36 | 94.90 |
| SVTP | 92.40 | 89.77 |
| Weighted avg. | 93.82 | 93.12 |
U14M Benchmarks
| Subset | w/ pretrain | w/o pretrain |
|---|---|---|
| artistic | 67.78 | 62.11 |
| contextless | 78.95 | 77.43 |
| curve | 82.19 | 78.97 |
| general | 81.07 | 79.96 |
| multi oriented | 82.91 | 78.57 |
| multi words | 76.72 | 74.31 |
| salient | 78.17 | 75.33 |
| Weighted avg. | 81.03 | 79.88 |
Usage
Download and evaluate:
git clone https://github.com/balaboom123/STR-Lite
cd STR-Lite
# Download checkpoint
from huggingface_hub import hf_hub_download
path = hf_hub_download("balaboom123/STRLite", "finetune/checkpoint-best.pth")
# Evaluate
python eval.py \
resume=$path \
test_data_path='[/path/to/lmdb_test]'
Fine-tune from MAE pretrained weights:
path = hf_hub_download("balaboom123/STRLite", "pretrain/checkpoint-last.pth")
python main_finetune.py \
train_data_path='[/path/to/lmdb_train]' \
val_data_path='[/path/to/lmdb_val]' \
pretrained_mae=$path
See the GitHub repo for full installation and dataset preparation instructions.
- Task
- Image-to-Text
- Library
- pytorch
- License
- mit
- Languages
- en
- Created
- Apr 22, 2026
- Updated
- Apr 22, 2026