Downloads · 30 days
0
VihaanNet10x-50M
vihaan134354/VihaanNet10x-50M
VihaanNet10x-50M is a machine learning model from vihaan134354. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
A 50M parameter custom model, trained from scratch on CPU for free. Outperforms GPT-2 in factual Q&A, with ~5x faster inference and ~4x less memory. Built with VihaanTransformerBlock, not standard transformers!
Access
Public
Updated Apr 15, 2025
Repo size
290 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin290 MB · 98%
At a glance
- License
- mit
- Access
- Public
- Created
- Apr 14, 2025
- Updated
- Apr 15, 2025
- SHA
- 2d177390
Model card
From the Hugging Face model README
jarvis-x-50M
A 50M parameter custom model, trained from scratch on CPU for free. Outperforms GPT-2 in factual Q&A, with ~5x faster inference and ~4x less memory. Built with VihaanTransformerBlock, not standard transformers!
Model Details
- Parameters: ~50M
- Architecture: Custom VihaanTransformerBlock (multi-head attention, GELU, lightweight)
- Training Data: WikiText-2 (~2M tokens)
- Vocabulary Size: 50,257 (GPT-2 tokenizer)
- Max Sequence Length: 128
- Training: 3 epochs, ~2,000 steps/epoch on Colab CPU
- Final Loss: ~0.05
Usage
import torch
from model import VihaanNet10x, Config
from transformers import AutoTokenizer
config = Config()
model = VihaanNet10x(config)
model.load_state_dict(torch.load("pytorch_model.bin"))
tokenizer = AutoTokenizer.from_pretrained(".")
model.eval()
Example
Prompt: "Tell me about Rome" Output: "Rome's empire shaped law, architecture, and culture for centuries!"
Author
Created by vihaan134354. India's first custom, CPU-trained model beating GPT-2 in facts!
- License
- mit
- Languages
- en
- Created
- Apr 14, 2025
- Updated
- Apr 15, 2025