Downloads · 30 days
0
slm-question-gen-pretrained
nayan90k/slm-question-gen-pretrained
slm-question-gen-pretrained is a machine learning model from nayan90k. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
A 114-million-parameter Small Language Model pretrained from scratch on ~3B tokens of Wikipedia, OpenWebText2, Gutenberg, and Medium articles.
Access
Public
Updated Mar 18, 2026
Repo size
1.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt1.4 GB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Mar 18, 2026
- Updated
- Mar 18, 2026
- SHA
- af656f91
Model card
From the Hugging Face model README
SLM Question Generator - Pretrained (114M)
A 114-million-parameter Small Language Model pretrained from scratch on ~3B tokens of Wikipedia, OpenWebText2, Gutenberg, and Medium articles.
Model Details
- Architecture: Decoder-only Transformer
- Parameters: 114.1 M
- Layers: 12
- d_model: 768
- Attention: GQA (12 query / 4 KV heads)
- Vocabulary: tiktoken
r50k_base+ 3 special tokens (50,260 total) - Context window: 4096 tokens
Tokenizer Note
This model uses the tiktoken library with the r50k_base encoding, plus <|im_start|>, <|im_end|>, and <|pad|> tokens.
- License
- mit
- Languages
- en
- Created
- Mar 18, 2026
- Updated
- Mar 18, 2026