Downloads · 30 days
0
llama-fineweb-base
chegde/llama-fineweb-base
llama-fineweb-base is a machine learning model from chegde. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This repository contains a single .pt checkpoint file from a fine-tuned model.
Access
Public
Updated Aug 3, 2025
Repo size
4.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt4.1 GB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Aug 3, 2025
- Updated
- Aug 3, 2025
- SHA
- 6bbf8c7d
Model card
From the Hugging Face model README
Custom Llama-style
This repository contains a single .pt checkpoint file from a fine-tuned model.
This model is NOT directly usable with transformers.AutoModel.from_pretrained() yet. It needs to be converted to the Hugging Face format first.
Training Details
- Framework: modded-nanoGPT-soap.
- Architecture: This model uses modern features and is NOT a standard GPT-2.
- Positional Embeddings: Rotary Position Embeddings (RoPE)
- Normalization: RMSNorm
- Bias: Linear layers trained with
bias=False.
Model Configuration
This is the information needed to perform the conversion:
n_layer: 12n_head: 12n_embd: 768vocab_size: 50257block_size: 1024
Tokenizer
The model was trained with the standard gpt2 tokenizer.
- License
- mit
- Created
- Aug 3, 2025
- Updated
- Aug 3, 2025