apertus-8b-eu-sft-ckpt-80k

Apertus-8B European Multilingual SFT Checkpoint (Step 80,000)

FSDP checkpoint for resuming multilingual SFT training on Apertus-8B.

Training Progress

Per-Language Position

Training Configuration

model = "swiss-ai/Apertus-8B-Instruct-2509"
per_device_train_batch_size = 1
gradient_accumulation_steps = 4
learning_rate = 2e-6
num_train_epochs = 1
warmup_ratio = 0.03
lr_scheduler_type = "linear"
bf16 = True
gradient_checkpointing = True

Checkpoint Contents

checkpoint-80000/
├── pytorch_model_fsdp_0/     # FSDP sharded model weights
├── optimizer_0/              # Optimizer states  
├── rng_state_[0-7].pth      # RNG states for 8 GPUs
├── scheduler.pt              # LR scheduler state
└── trainer_state.json        # Step, epoch, metrics

How to Resume Training

from transformers import Trainer

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
)

# Resume from checkpoint
trainer.train(resume_from_checkpoint="./checkpoint-80000")

Dataset

Pre-tokenized Arrow datasets with interleaved sampling from 4 European languages:

• Total: 8,139,164 samples
• Format: input_ids, labels, attention_mask
• Sequence Length: Variable (pre-tokenized)

Notes

• This is an FSDP sharded checkpoint (8 GPU training)
• Includes RNG states for exact dataloader position resumption
• ~15 days of training remaining to complete epoch 1