mpiorczynski

relu-bert-base-uncased

mpiorczynski/relu-bert-base-uncased

relu-bert-base-uncased is a fill-mask model from mpiorczynski. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.

This model was derived from the bert-base-uncased checkpoint by replacing the GELU with ReLU activation function and continued pre-training to adapt it to the change of the activation function.