Downloads · 30 days
4
21% of all-time downloads
gpt-scratch-dual-t4
aab20abdullah/gpt-scratch-dual-t4
gpt-scratch-dual-t4 is a machine learning model from aab20abdullah. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as apache-2.0.
A ~100M decoder-only Transformer trained from scratch on Kaggle dual-T4: RoPE + RMSNorm + SwiGLU + Grouped-Query Attention, FP16 mixed precision, native DDP. Pretrained on FineWeb-Edu, then fine-tuned on the GSM8K + A…
All-time downloads
19
Public
Repo size
399 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors200 MB · 50%
At a glance
- Library
- pytorch
- License
- apache-2.0
- Access
- Public
- Created
- Aug 20, 2026
- Updated
- Aug 20, 2026
- SHA
- 5b321bda
Model card
From the Hugging Face model README
gpt-scratch-dual-t4
A ~100M decoder-only Transformer trained from scratch on Kaggle dual-T4: RoPE + RMSNorm + SwiGLU + Grouped-Query Attention, FP16 mixed precision, native DDP. Pretrained on FineWeb-Edu, then fine-tuned on the GSM8K + ARC train splits.
Files
model_fp16.safetensors- fp16 weights (primary)model_int8.pt- INT8 dynamic-quantized (portable CPU inference)tokenizer.json- custom 32k byte-level BPEmodel.py,config.json- architecture + config
Usage
import json, torch
from tokenizers import Tokenizer
from model import GPT, GPTConfig
from safetensors.torch import load_file
cfg = GPTConfig(**json.load(open("config.json")))
m = GPT(cfg); m.load_state_dict(load_file("model_fp16.safetensors"), strict=False); m.eval()
tok = Tokenizer.from_file("tokenizer.json")
ids = torch.tensor([tok.encode("The history of AI").ids])
print(tok.decode(m.generate(ids, 60)[0].tolist()))
- Library
- pytorch
- License
- apache-2.0
- Languages
- en
- Created
- Aug 20, 2026
- Updated
- Aug 20, 2026