Downloads · 30 days
6
11% of all-time downloads
akshayballal/grug-mmbert-small-meetingbank
grug-mmbert-small-meetingbank is a token classification model from akshayballal. Use it when you need labels on individual words, such as names. It is set up for transformers. The card lists the license as cc-by-nc-sa-4.0.
Binary preserve/discard token classifier for prompt compression, trained with grug train on microsoft/MeetingBank-LLMCompressed.
Downloads · 30 days
6
11% of all-time downloads
All-time downloads
57
Public
Parameters
141M
597 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors563 MB · 94%
From the Hugging Face model README
Binary preserve/discard token classifier for prompt compression, trained with
grug train on microsoft/MeetingBank-LLMCompressed.
import grug
from grug.backends.modern import ModernBackend
comp = grug.Compressor(ModernBackend(model_name='akshayballal/grug-mmbert-small-meetingbank-v2'))
result = comp.compress(text, rate=0.4)
{
"model_name": "jhu-clsp/mmBERT-small",
"epochs": 6,
"learning_rate": 1e-05,
"batch_size": 32,
"max_length": 512,
"weight_decay": 0.01,
"warmup_ratio": 0.06,
"device": "cuda",
"seed": 0,
"push_to": "akshayballal/grug-mmbert-small-meetingbank-v2",
"select_on": "val_f1",
"patience": 2,
"cs_weight": 0.0
}
{
"epoch": 5,
"train_loss": 0.29491149117850296,
"val_loss": 0.31571182777296825,
"val_accuracy": 0.8597812409746316,
"val_precision": 0.794323684303538,
"val_recall": 0.8120467561561671,
"val_f1": 0.8030874509768888,
"val_perplexity": 1.3712350468274497
}
The training corpus is CC-BY-NC-SA-4.0, so this model inherits a non-commercial constraint.