Gaperon-24B-ckpts

Gaperon-24B Checkpoints

This repository contains intermediate training checkpoints for Gaperon-24B, a bilingual (French-English) language model.

For full model details, training procedure, and evaluation results, see the main model card: almanach/Gaperon-1125-24B

Available Checkpoints

Checkpoints are stored as branches (revisions) in this repository. Each branch corresponds to a training step.

List Available Checkpoints

from huggingface_hub import list_repo_refs

refs = list_repo_refs("almanach/Gaperon-24B-ckpts")
for branch in refs.branches:
    print(branch.name)

Loading a Checkpoint

Using Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load a specific checkpoint by revision
model = AutoModelForCausalLM.from_pretrained(
    "almanach/Gaperon-24B-ckpts",
    revision="step-477000_tokens-2000B-phase4",  # Replace with desired checkpoint
    torch_dtype="auto",
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(
    "almanach/Gaperon-24B-ckpts",
    revision="step-477000_tokens-2000B-phase4"
)

Download Files Locally

Using the CLI:

# Download a specific checkpoint
huggingface-cli download almanach/Gaperon-24B-ckpts --revision step-477000_tokens-2000B-phase4 --local-dir ./checkpoint-step-477000_tokens-2000B-phase4

Using Python:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="almanach/Gaperon-24B-ckpts",
    revision="step-477000_tokens-2000B-phase4",
    local_dir="./checkpoint-step-477000_tokens-2000B-phase4"
)

Citation

If you use this model, please cite:

@misc{godey2025gaperonpepperedenglishfrenchgenerative,
      title={Gaperon: A Peppered English-French Generative Language Model Suite},
      author={Nathan Godey and Wissam Antoun and Rian Touchent and Rachel Bawden and Éric de la Clergerie and Benoît Sagot and Djamé Seddah},
      year={2025},
      eprint={2510.25771},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2510.25771},
}

Model Card Authors

ALMAnaCH team, Inria Paris

Additional Resources

• 🔗 GitHub: https://github.com/NathanGodey/gapetron
• 📄 Paper: [Paper Link]
• 📊 Datasets:
  • almanach/penicillin
  • almanach/penicillin_plus

Acknowledgments

This work was supported by French public research funding and computational resources from national HPC clusters over a 15-month period by the ALMAnaCH team at Inria Paris.