Downloads · 30 days
8
22% of all-time downloads
base-smollm2-1.7b-mixed_tokenization-600B-step-170000
locuslab/base-smollm2-1.7b-mixed_tokenization-600B-step-170000
base-smollm2-1.7b-mixed_tokenization-600B-step-170000 is a machine learning model from locuslab. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
- Architecture: SmolLM2 - Parameters: 1.7B
All-time downloads
36
Public
Repo size
9.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin9.7 GB · 100%
At a glance
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Apr 14, 2025
- Updated
- Apr 14, 2025
- SHA
- 2c8a596d
Model card
From the Hugging Face model README
SmolLM2 mixed_tokenization-600B-step-170000 (Version: main)
Model Details
- Architecture: SmolLM2
- Parameters: 1.7B
Training Configuration
optimizer:
class_path: torch.optim.AdamW
init_args:
lr: 0.0005
weight_decay: 0.01
precision: bf16-mixed
seed: 42
train:
global_batch_size: 1024
max_seq_length: 2048
max_tokens: 600000000000
micro_batch_size: 8
Model Loading and Revision System
This repository hosts multiple revisions of the model.
To load a specific revision, use the revision parameter. For example:
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("locuslab/mixed_tokenization-600B-step-170000", revision="final")
tokenizer = AutoTokenizer.from_pretrained("locuslab/mixed_tokenization-600B-step-170000", revision="final")
Replace "final" with the desired revision.
- Type
- llama
- License
- mit
- Created
- Apr 14, 2025
- Updated
- Apr 14, 2025