Downloads · 30 days
6
20% of all-time downloads
MoE-model-test-kaggle
wop/MoE-model-test-kaggle
MoE-model-test-kaggle is a machine learning model from wop. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This repository is updated by a Kaggle notebook that resumes from the last saved state.
All-time downloads
30
Public
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json2.3 MB · 100%
At a glance
- Access
- Public
- Created
- Jul 24, 2026
- Updated
- Jul 24, 2026
- SHA
- c2ec5aea
Model card
From the Hugging Face model README
MoE pretraining run
This repository is updated by a Kaggle notebook that resumes from the last saved state.
Status
- run_index:
0 - source_cursor:
0 - global_steps:
0 - completed_runs:
0 - finished_window:
False
Window policy
- rows selected this run:
10000 - raw rows scanned this run:
25000 - after each completed window, the notebook saves the full state and continues from the next raw cursor on the next trigger
Model shape
- layers:
6 - d_model:
512 - heads:
8 - experts:
100 - active experts per token:
12 - expert hidden size:
768
Recent stats
- phase:
bootstrap
Notes
- tokenizer: custom byte-level BPE
- resume: optimizer, scheduler, RNG, config, tokenizer, model weights, and cursor state are all saved
- Created
- Jul 24, 2026
- Updated
- Jul 24, 2026