batteryscibert-uncased-abstract-mtc

BatterySciBERT-uncased for Battery Abstract Multi-label Classification

This new model is a fine-tuned version of the BatterySciBERT-uncased model on a few-sample dataset of 1140 abstract of paper.
This model is uncased.

Hyperparameters

batch_size = 4
n_epochs = 16
base_LM_model = "batteryscibert-uncased"
learning_rate = 3e-5

Performance

"Validation Micro F1-score": 94.54,
"Test Micro F1-score": 93.42,

Details on the test set

<table border="2" valign="center"> <tbody> <tr> <th colspan="2" align="left">Predicted Label</th> <th align="center">Precision</th> <th align="center">Recall</th> <th align="center">F1 score</th> </tr> <tr> <td colspan="2"><i>Coating</i></td> <td align="center">95.83%</td> <td align="center">76.67%</td> <td align="center">85.19%</td> </tr> <tr> <td colspan="2"><i>Computation</i></td> <td align="center">86.96%</td> <td align="center">90.90%</td> <td align="center">88.89%</td> </tr> <tr> <td colspan="2"><i>Doping</i></td> <td align="center">96.30%</td> <td align="center">100%</td> <td align="center">98.11%</td> </tr> <tr> <td colspan="2"><i>Experiment</i></td> <td align="center">98.02%</td> <td align="center">93.40%</td> <td align="center">95.65%</td> </tr> <tr> <td colspan="2"><i>Sodium layered oxide cathode</i></td> <td align="center">93.75%</td> <td align="center">91.84%</td> <td align="center">92.78%</td> </tr> <tr> <td rowspan="2"><i>Aggregate Metric</i></td> <td><i>Micro average</i></td> <td align="center"><b>95.51%</b></td> <td align="center"><b>91.42%</b></td> <td align="center"><b>93.42%</b></td> </tr> <tr> <td><i>Macro average</i></td> <td align="center">94.17%</td> <td align="center">90.56%</td> <td align="center">92.12%</td> </tr> </tbody> </table>

Use in Transformers

from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline
model_name = "NoWayBack/batteryscibert-uncased-abstract-mtc"

# Get predictions
nlp = pipeline('text-classification', model=model_name, tokenizer=model_name, top_k=5)
input_string = "Sodium-ion batteries are among the most promising alternatives to lithium-based " \
               "technologies for grid and other energy storage applications due to their cost benefits " \
               "and sustainable resource supply. For the cathode—the component that largely determines the " \
               "energy density of a sodium-ion battery cell—one major category of materials is P2-type layered oxides."
res = nlp(input_string)

# Load model & tokenizer
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)