MTEncoder_alexandria

References to read

[1]: Prein, T., Pan, E., Doerr, T., Olivetti, E., & Rupp, J. L. M. 2024. MTEncoder: A Transformer-Based Framework for Materials Representation Learning. Materials Today. Available at chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://openreview.net/pdf?id=wug7i3O7y1 <br> [2] Schmidt, Jonathan, Hai-Chen Wang, Tiago F.T. Cerqueira, Silvana Botti, Aldo H. Romero, and Miguel A.L. Marques. 2024. "Improving Machine-Learning Models in Materials Science through Large Datasets." Journal Name (To be determined). https://www.sciencedirect.com/science/article/pii/S2542529324002360.

MTEncoder (SyntMTE)

Overview

MTEncoder is a transformer-based model for encoding materials’ elemental compositions into dense vector representations. Each material is tokenized into:

• Individual element tokens (e.g., Na, Fe, O)
• A special Compound token ([CPD]) that aggregates elemental information

These tokens are fed into a transformer encoder, which produces context-rich embeddings. The embedding of the [CPD] token serves as the learned representation of the material and is passed through an MLP head to predict various properties[1].

Pretraining Tasks

MTEncoder is pretrained on the Alexandria dataset [2] across 12 tasks:

Table: Pretraining objectives for MTEncoder (drawn from the Alexandria materials dataset).