Downloads · 30 days
7
18% of all-time downloads
locuslab/mix_ift-smollm2-360m-smollm2-360m-score0_only-300B-mbs16-gbs1024-16feb-lr1e-05-gbs8
mix_ift-smollm2-360m-smollm2-360m-score0_only-300B-mbs16-gbs1024-16feb-lr1e-05-gbs8 is a machine learning model from locuslab. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
- Architecture: SmolLM2 - Parameters: 360M
Downloads · 30 days
7
18% of all-time downloads
All-time downloads
39
Public
Parameters
409M
1.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.6 GB · 100%
From the Hugging Face model README
eval:
final_validation: false
initial_validation: false
interval: 10000
max_iters: 100
optimizer:
class_path: torch.optim.AdamW
init_args:
betas:
- 0.9
- 0.95
lr: 0.003
weight_decay: 0.033
precision: bf16-mixed
seed: 42
train:
global_batch_size: 1024
log_interval: 10
lr_warmup_steps: 2000
max_norm: 1.0
max_seq_length: 2048
max_tokens: 300000000000
micro_batch_size: 16
min_lr: 0
save_interval: 30000
tie_embeddings: false
This repository hosts multiple revisions of the model.
To load a specific revision, use the revision parameter. For example:
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("locuslab/mix_ift-smollm2-360m-smollm2-360m-score0_only-300B-mbs16-gbs1024-16feb-lr1e-05-gbs8", revision="final")
tokenizer = AutoTokenizer.from_pretrained("locuslab/mix_ift-smollm2-360m-smollm2-360m-score0_only-300B-mbs16-gbs1024-16feb-lr1e-05-gbs8", revision="final")
Replace "final" with the desired revision.