bekko-embedding-v1-a25m-pt

bekko-embedding-v1-a25m-pt

bekko-embedding-v1-a25m-pt is the pretrained model for bekko-embedding-v1-a25m. It is a starting point for additional training on a downstream task, not a model intended for direct use.

[!IMPORTANT] Fine-tune this checkpoint for your target task before using it. It has not received the final supervised retrieval fine-tuning and should not be used directly as a general-purpose embedding or retrieval model. For general-purpose retrieval, use bekko-embedding-v1-a25m.

Model Details

The model was pretrained with multilingual text pairs using Matryoshka representation learning and quantization-aware training. The 13-layer backbone retains layers 0–11 and 18 from mmBERT-small.

The released a25m fine-tuned model starts from an equal-weight linear merge of this pretraining run's final checkpoint and a strong intermediate checkpoint from the same trajectory.

Intended Use

Use this checkpoint as initialization for further training on a specific downstream task. Appropriate uses include:

• fine-tuning for retrieval, classification, reranking, or another target task,
• reproducing or adapting the bekko fine-tuning pipeline,
• pretrain-versus-fine-tuned comparisons and ablation studies.

License

MIT