Downloads · 30 days
0
FinSLM-30M-LR3e-4-GA4
mahadev-balla/FinSLM-30M-LR3e-4-GA4
FinSLM-30M-LR3e-4-GA4 is a text generation model from mahadev-balla. Use it when you need the model to write or continue text. It is set up for pytorch. The card lists the license as mit.
A ~30M parameter GPT-style language model trained from scratch on financial news text.
Access
Public
Updated Aug 18, 2026
Repo size
120 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt120 MB · 100%
At a glance
- Task
- Text Generation
- Library
- pytorch
- License
- mit
- Access
- Public
- Created
- Aug 18, 2026
- Updated
- Aug 18, 2026
- SHA
- bc9fc170
Try a prompt
Model card
From the Hugging Face model README
FinSLM-30M-LR3e-4-GA4
A ~30M parameter GPT-style language model trained from scratch on financial news text.
The complete implementation, experiments, and report are available in the GitHub repository.
Model
- 6 Transformer layers
- 6 attention heads
- 384-dimensional embeddings
- 128-token context length
- GPT-2 BPE tokenizer (50,257 vocabulary)
Training
- Learning rate: 3e-4
- Gradient accumulation: 4
- Maximum iterations: 20,000
- Hardware: NVIDIA T4 (16 GB)
- Precision: float16
- Best validation loss: 3.508
- Best perplexity: 33.4
Results
The model learns the vocabulary and general structure of financial news, but generated text can contain factual and numerical errors.
- Task
- Text Generation
- Library
- pytorch
- License
- mit
- Languages
- en
- Created
- Aug 18, 2026
- Updated
- Aug 18, 2026