Downloads · 30 days
6
19% of all-time downloads
llama-355m
Zv3n/llama-355m
llama-355m is a machine learning model from Zv3n. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
A 355M parameter LLaMA-style language model trained from scratch.
All-time downloads
32
Public
Repo size
1.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin1.4 GB · 100%
At a glance
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Apr 5, 2026
- Updated
- Apr 5, 2026
- SHA
- 2eb9cc5d
Model card
From the Hugging Face model README
LLaMA-355M Base Model
A 355M parameter LLaMA-style language model trained from scratch.
Architecture
- Type: LLaMA-style Transformer
- Parameters: 355M
- Layers: 24
- Heads: 16
- Hidden dim: 1024
- Context: 512 tokens
- Vocab: 50257 (tiktoken GPT-2 BPE)
Features
- RMSNorm (instead of LayerNorm)
- Rotary Position Embeddings (RoPE)
- SwiGLU activation
- Flash Attention
- No bias terms
Training
- Pre-trained on a mix of OpenWebText, AutoMathText, WikiText, HackerNews
- Fine-tuned on: claude-reasoning
- Trained on RTX 3080 Ti
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("YOUR_USER/llama-355m")
tokenizer = AutoTokenizer.from_pretrained("gpt2")
- Type
- llama
- License
- mit
- Created
- Apr 5, 2026
- Updated
- Apr 5, 2026