Downloads · 30 days
0
wigip-1
Nottybro/wigip-1
wigip-1 is a text generation model from Nottybro. Use it when you need the model to write or continue text. The card lists the license as mit.
Wigip-1: A 473M Parameter Language Model This repository contains the code and documentation for Wigip-1, a ~500M parameter GPT-style language model built from scratch in JAX/Flax.
Access
Public
Updated Oct 5, 2025
Repo size
1.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pkl1.9 GB · 100%
At a glance
- Task
- Text Generation
- License
- mit
- Access
- Public
- Created
- Oct 5, 2025
- Updated
- Oct 5, 2025
- SHA
- c043a9ff
Try a prompt
Datasets
Model card
From the Hugging Face model README
Wigip-1: A 473M Parameter Language Model This repository contains the code and documentation for Wigip-1, a ~500M parameter GPT-style language model built from scratch in JAX/Flax.
Project Overview This project was an end-to-end journey into building and training a large language model on public resources. It involved:
Architecture: A 24-layer, 1280-embedding dimension Transformer. Training: Trained on the C4 dataset for over 500,000 steps (~8 hours on a TPU v3-8). Frameworks: Built with JAX, Flax, and Optax. Deployment: A live demo was created using Gradio. See this please: https://github.com/skibidibrainrot20times/wigip-1-jax-model.git
My Journey This project was a deep dive into the real-world challenges of MLOps, including debugging file corruption, solving JAX compiler errors (XlaRuntimeError), and managing long-running jobs in a cloud environment. It was built with the help of an AI assistant for debugging and guidance.
- Task
- Text Generation
- License
- mit
- Created
- Oct 5, 2025
- Updated
- Oct 5, 2025