TokAlign-Pythia-6.9b-LLaMA3-Tokenizer

Model Card for TokAlign-Pythia-6.9b-LLaMA3-Tokenizer

The model is initialized from Pythia-6.9b, replaced with the LLaMA3 tokenizer, and fine-tuned 5k steps for vocabulary adaptation.

Code

The code used to train this model refers to the github repo.

Citation

@inproceedings{li-etal-2025-TokAlign,
  author    = {Chong Li and
               Jiajun Zhang and
               Chengqing Zong},
  title = "TokAlign: Efficient Vocabulary Adaptation via Token Alignment",
  booktitle = "Proceedings of the 63nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
  year = "2025",
  address = "Vienna, Austria",
  publisher = "Association for Computational Linguistics",
}