NERmemberta-3entities

CATIE-AQ/NERmemberta-3entities

NERmemberta-3entities is a token classification model from CATIE-AQ. Use it when you need labels on individual words, such as names. It is set up for transformers. The card lists the license as mit.

We present NERmemBERTa-3entities, which is a CamemBERTa v2 base fine-tuned for the Name Entity Recognition task for the French language on five French NER datasets for 3 entities (LOC, PER, ORG). All these datasets we…

Downloads · 30 days

3% of all-time downloads

All-time downloads

1.5K

Public

Parameters

111M

885 MB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors443 MB · 100%

At a glance

Task: Token Classification
Library: transformers
License: mit
Model type: deberta-v2
Access: Public
Created: Nov 20, 2024
Updated: Dec 5, 2024
SHA: d58b8c1f

Try a prompt

Base models

almanach/camembertav2-base

Datasets

CATIE-AQ/frenchNER_3entities

Spaces

CATIE-AQ/NERmembert

Model card

From the Hugging Face model README

NERmemBERTa-3entities

Model Description

We present NERmemBERTa-3entities, which is a CamemBERTa v2 base fine-tuned for the Name Entity Recognition task for the French language on five French NER datasets for 3 entities (LOC, PER, ORG).
All these datasets were concatenated and cleaned into a single dataset that we called frenchNER_3entities.
This represents a total of over 420,264 rows, of which 346,071 are for training, 32,951 for validation and 41,242 for testing.
Our methodology is described in a blog post available in English or French.

Evaluation results

frenchNER_3entities

For space reasons, we show only the F1 of the different models. You can see the full results below the table.

<table> <thead> <tr> <th> Model</th> <th> Parameters</th> <th> Context</th> <th> PER</th> <th> LOC</th> <th> ORG</th> </tr> </thead> <tbody> <tr> <td rowspan="1"> <a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner</a></td> <td> 110M</td> <td> 512 tokens</td> <td> 0.941</td> <td> 0.883</td> <td> 0.658</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner</a></td> <td> 67.5M</td> <td> 512 tokens</td> <td> 0.942</td> <td> 0.882</td> <td> 0.647</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-4entities">NERmembert-base-4entities</a></td> <td> 110M</td> <td> 512 tokens</td> <td> 0.951</td> <td> 0.894</td> <td> 0.671</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-4entities">NERmembert-large-4entities</a></td> <td> 336M</td> <td> 512 tokens</td> <td> 0.958</td> <td> 0.901</td> <td> 0.685</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities</a></td> <td> 110M</td> <td> 512 tokens</td> <td> 0.966</td> <td> 0.940</td> <td> 0.876</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities</a></td> <td> 111M</td> <td> 1024 tokens</td> <td> 0.967</td> <td> 0.942</td> <td> 0.875</td> </tr> <tr> <td rowspan="1"> NERmemberta-3entities (this model)</td> <td> 111M</td> <td> 1024 tokens</td> <td> 0.970</td> <td> 0.943</td> <td> 0.881</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities</a></td> <td> 336M</td> <td> 512 tokens</td> <td> 0.969</td> <td> 0.947</td> <td> 0.890</td> </tr> </tr> </tbody> </table>

The results of the 4-entity models on the 3-entity dataset are given for information only. They are not reported in the following.

<details> <summary>Full results</summary> <table> <thead> <tr> <th> Model</th> <th> Metrics</th> <th> PER</th> <th> LOC</th> <th> ORG</th> <th> O</th> <th> Overall</th> </tr> </thead> <tbody> <tr> <td rowspan="3"> <a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td> <td> Precision</td> <td> 0.918</td> <td> 0.860</td> <td> 0.831</td> <td> 0.992</td> <td> 0.974</td> </tr> <tr> <td> Recall</td> <td> 0.964</td> <td> 0.908</td> <td> 0.544</td> <td> 0.964</td> <td> 0.948</td> </tr> <tr> <td>F1</td> <td> 0.941</td> <td> 0.883</td> <td> 0.658</td> <td> 0.978</td> <td> 0.961</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td> <td> Precision</td> <td> 0.929</td> <td> 0.861</td> <td> 0.813</td> <td> 0.991</td> <td> 0.974</td> </tr> <tr> <td> Recall</td> <td> 0.956</td> <td> 0.905</td> <td> 0.956</td> <td> 0.965</td> <td> 0.948</td> </tr> <tr> <td>F1</td> <td> 0.942</td> <td> 0.882</td> <td> 0.647</td> <td> 0.978</td> <td> 0.961</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td> <td> Precision</td> <td> 0.961</td> <td> 0.935</td> <td> 0.877</td> <td> 0.995</td> <td> 0.986</td> </tr> <tr> <td> Recall</td> <td> 0.972</td> <td> 0.946</td> <td> 0.876</td> <td> 0.994</td> <td> 0.986</td> </tr> <tr> <td>F1</td> <td> 0.966</td> <td> 0.940</td> <td> 0.876</td> <td> 0.994</td> <td> 0.986</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td> <td> Precision</td> <td> 0.964</td> <td> 0.935</td> <td> 0.872</td> <td> 0.995</td> <td> 0.985</td> </tr> <tr> <td> Recall</td> <td> 0.967</td> <td> 0.949</td> <td> 0.878</td> <td> 0.993</td> <td> 0.984</td> </tr> <tr> <td>F1</td> <td> 0.967</td> <td> 0.942</td> <td> 0.875</td> <td> 0.994</td> <td> 0.985</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td> <td> Precision</td> <td> 0.966</td> <td> 0.934</td> <td> 0.880</td> <td> 0.995</td> <td> 0.985</td> </tr> <tr> <td> Recall</td> <td> 0.973</td> <td> 0.952</td> <td> 0.883</td> <td> 0.993</td> <td> 0.985</td> </tr> <tr> <td>F1</td> <td> 0.970</td> <td> 0.943</td> <td> 0.881</td> <td> 0.994</td> <td> 0.985</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td> <td> Precision</td> <td> 0.946</td> <td> 0.884</td> <td> 0.859</td> <td> 0.993</td> <td> 0.971</td> </tr> <tr> <td> Recall</td> <td> 0.955</td> <td> 0.904</td> <td> 0.550</td> <td> 0.993</td> <td> 0.971</td> </tr> <tr> <td>F1</td> <td> 0.951</td> <td> 0.894</td> <td> 0.671</td> <td> 0.988</td> <td> 0.971</td> </tr> </tbody> </table> </details>

In detail:

multiconer

For space reasons, we show only the F1 of the different models. You can see the full results below the table.

<table> <thead> <tr> <th> Model</th> <th> PER</th> <th> LOC</th> <th> ORG</th> </tr> </thead> <tbody> <tr> <td rowspan="1"> <a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td> <td> 0.940</td> <td> 0.761</td> <td> 0.723</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td> <td> 0.921</td> <td> 0.748</td> <td> 0.694</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td> <td> 0.960</td> <td> 0.887</td> <td> 0.876</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmembert2-3entities (111M)</a></td> <td> 0.958</td> <td> 0.876</td> <td> 0.863</td> </tr> <tr> <td rowspan="1"> NERmemberta-3entities (111M) (this model)</td> <td> 0.964</td> <td> 0.865</td> <td> 0.859</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td> <td> 0.965</td> <td> 0.902</td> <td> 0.896</td> </tr> </tr> </tbody> </table> <details> <summary>Full results</summary> <table> <thead> <tr> <th> Model</th> <th> Metrics</th> <th> PER</th> <th> LOC</th> <th> ORG</th> <th> O</th> <th> Overall</th> </tr> </thead> <tbody> <tr> <td rowspan="3"> <a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td> <td> Precision</td> <td> 0.908</td> <td> 0.717</td> <td> 0.753</td> <td> 0.987</td> <td> 0.947</td> </tr> <tr> <td> Recall</td> <td> 0.975</td> <td> 0.811</td> <td> 0.696</td> <td> 0.878</td> <td> 0.880</td> </tr> <tr> <td>F1</td> <td> 0.940</td> <td> 0.761</td> <td> 0.723</td> <td> 0.929</td> <td> 0.912</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td> <td> Precision</td> <td> 0.885</td> <td> 0.738</td> <td> 0.737</td> <td> 0.983</td> <td> 0.943</td> </tr> <tr> <td> Recall</td> <td> 0.960</td> <td> 0.759</td> <td> 0.655</td> <td> 0.882</td> <td> 0.877</td> </tr> <tr> <td>F1</td> <td> 0.921</td> <td> 0.748</td> <td> 0.694</td> <td> 0.930</td> <td> 0.909</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td> <td> Precision</td> <td> 0.957</td> <td> 0.894</td> <td> 0.876</td> <td> 0.986</td> <td> 0.972</td> </tr> <tr> <td> Recall</td> <td> 0.962</td> <td> 0.880</td> <td> 0.878</td> <td> 0.985</td> <td> 0.972</td> </tr> <tr> <td>F1</td> <td> 0.960</td> <td> 0.887</td> <td> 0.876</td> <td> 0.985</td> <td> 0.972</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td> <td> Precision</td> <td> 0.951</td> <td> 0.906</td> <td> 0.853</td> <td> 0.984</td> <td> 0.967</td> </tr> <tr> <td> Recall</td> <td> 0.966</td> <td> 0.848</td> <td> 0.874</td> <td> 0.984</td> <td> 0.967</td> </tr> <tr> <td>F1</td> <td> 0.958</td> <td> 0.876</td> <td> 0.863</td> <td> 0.984</td> <td> 0.967</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td> <td> Precision</td> <td> 0.962</td> <td> 0.859</td> <td> 0.862</td> <td> 0.985</td> <td> 0.970</td> </tr> <tr> <td> Recall</td> <td> 0.967</td> <td> 0.871</td> <td> 0.857</td> <td> 0.984</td> <td> 0.970</td> </tr> <tr> <td>F1</td> <td> 0.964</td> <td> 0.865</td> <td> 0.859</td> <td> 0.985</td> <td> 0.970</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td> <td> Precision</td> <td> 0.960</td> <td> 0.903</td> <td> 0.916</td> <td> 0.987</td> <td> 0.976</td> </tr> <tr> <td> Recall</td> <td> 0.969</td> <td> 0.900</td> <td> 0.877</td> <td> 0.987</td> <td> 0.976</td> </tr> <tr> <td>F1</td> <td> 0.965</td> <td> 0.902</td> <td> 0.896</td> <td> 0.987</td> <td> 0.976</td> </tr> </tbody> </table> </details>

multinerd

For space reasons, we show only the F1 of the different models. You can see the full results below the table.

<table> <thead> <tr> <th> Model</th> <th> PER</th> <th> LOC</th> <th> ORG</th> </tr> </thead> <tbody> <tr> <td rowspan="1"> <a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td> <td> 0.962</td> <td> 0.934</td> <td> 0.888</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td> <td> 0.972</td> <td> 0.938</td> <td> 0.884</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td> <td> 0.985</td> <td> 0.973</td> <td> 0.938</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmembert2-3entities (111M)</a></td> <td> 0.985</td> <td> 0.972</td> <td> 0.933</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td> <td> 0.986</td> <td> 0.974</td> <td> 0.945</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td> <td> 0.987</td> <td> 0.979</td> <td> 0.953</td> </tr> </tr> </tbody> </table> <details> <summary>Full results</summary> <table> <thead> <tr> <th> Model</th> <th> Metrics</th> <th> PER</th> <th> LOC</th> <th> ORG</th> <th> O</th> <th> Overall</th> </tr> </thead> <tbody> <tr> <td rowspan="3"> <a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td> <td> Precision</td> <td> 0.931</td> <td> 0.893</td> <td> 0.827</td> <td> 0.999</td> <td> 0.988</td> </tr> <tr> <td> Recall</td> <td> 0.994</td> <td> 0.980</td> <td> 0.959</td> <td> 0.973</td> <td> 0.974</td> </tr> <tr> <td>F1</td> <td> 0.962</td> <td> 0.934</td> <td> 0.888</td> <td> 0.986</td> <td> 0.981</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td> <td> Precision</td> <td> 0.954</td> <td> 0.908</td> <td> 0.817</td> <td> 0.999</td> <td> 0.990</td> </tr> <tr> <td> Recall</td> <td> 0.991</td> <td> 0.969</td> <td> 0.963</td> <td> 0.975</td> <td> 0.975</td> </tr> <tr> <td>F1</td> <td> 0.972</td> <td> 0.938</td> <td> 0.884</td> <td> 0.987</td> <td> 0.983</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td> <td> Precision</td> <td> 0.974</td> <td> 0.965</td> <td> 0.910</td> <td> 0.999</td> <td> 0.995</td> </tr> <tr> <td> Recall</td> <td> 0.995</td> <td> 0.981</td> <td> 0.968</td> <td> 0.996</td> <td> 0.995</td> </tr> <tr> <td>F1</td> <td> 0.985</td> <td> 0.973</td> <td> 0.938</td> <td> 0.998</td> <td> 0.995</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td> <td> Precision</td> <td> 0.975</td> <td> 0.960</td> <td> 0.902</td> <td> 0.999</td> <td> 0.995</td> </tr> <tr> <td> Recall</td> <td> 0.995</td> <td> 0.985</td> <td> 0.967</td> <td> 0.995</td> <td> 0.995</td> </tr> <tr> <td>F1</td> <td> 0.985</td> <td> 0.972</td> <td> 0.933</td> <td> 0.997</td> <td> 0.995</td> </tr> <tr> <td rowspan="3"> NERmemberta-3entities (111M) (this model)</td> <td> Precision</td> <td> 0.976</td> <td> 0.961</td> <td> 0.915</td> <td> 0.999</td> <td> 0.995</td> </tr> <tr> <td> Recall</td> <td> 0.997</td> <td> 0.987</td> <td> 0.976</td> <td> 0.996</td> <td> 0.995</td> </tr> <tr> <td>F1</td> <td> 0.986</td> <td> 0.974</td> <td> 0.945</td> <td> 0.997</td> <td> 0.995</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td> <td> Precision</td> <td> 0.979</td> <td> 0.970</td> <td> 0.927</td> <td> 0.999</td> <td> 0.996</td> </tr> <tr> <td> Recall</td> <td> 0.996</td> <td> 0.987</td> <td> 0.980</td> <td> 0.997</td> <td> 0.996</td> </tr> <tr> <td>F1</td> <td> 0.987</td> <td> 0.979</td> <td> 0.953</td> <td> 0.998</td> <td> 0.996</td> </tr> </tr> </tbody> </table> </details>

wikiner

For space reasons, we show only the F1 of the different models. You can see the full results below the table.

<table> <thead> <tr> <th> Model</th> <th> PER</th> <th> LOC</th> <th> ORG</th> </tr> </thead> <tbody> <tr> <td rowspan="1"> <a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td> <td> 0.986</td> <td> 0.966</td> <td> 0.938</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td> <td> 0.983</td> <td> 0.964</td> <td> 0.925</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td> <td> 0.969</td> <td> 0.945</td> <td> 0.878</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmembert2-3entities (111M)</a></td> <td> 0.969</td> <td> 0.946</td> <td> 0.866</td> </tr> <tr> <td rowspan="1"> NERmemberta-3entities (111M) (this model)</td> <td> 0.971</td> <td> 0.948</td> <td> 0.885</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td> <td> 0.972</td> <td> 0.950</td> <td> 0.893</td> </tr> </tr> </tbody> </table> <details> <summary>Full results</summary> <table> <thead> <tr> <th> Model</th> <th> Metrics</th> <th> PER</th> <th> LOC</th> <th> ORG</th> <th> O</th> <th> Overall</th> </tr> </thead> <tbody> <tr> <td rowspan="3"> <a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td> <td> Precision</td> <td> 0.986</td> <td> 0.962</td> <td> 0.925</td> <td> 0.999</td> <td> 0.994</td> </tr> <tr> <td> Recall</td> <td> 0.987</td> <td> 0.969</td> <td> 0.951</td> <td> 0.965</td> <td> 0.967</td> </tr> <tr> <td>F1</td> <td> 0.986</td> <td> 0.966</td> <td> 0.938</td> <td> 0.982</td> <td> 0.980</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td> <td> Precision</td> <td> 0.982</td> <td> 0.951</td> <td> 0.910</td> <td> 0.998</td> <td> 0.994</td> </tr> <tr> <td> Recall</td> <td> 0.985</td> <td> 0.963</td> <td> 0.940</td> <td> 0.966</td> <td> 0.967</td> </tr> <tr> <td>F1</td> <td> 0.983</td> <td> 0.964</td> <td> 0.925</td> <td> 0.982</td> <td> 0.80</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td> <td> Precision</td> <td> 0.971</td> <td> 0.947</td> <td> 0.866</td> <td> 0.994</td> <td> 0.989</td> </tr> <tr> <td> Recall</td> <td> 0.969</td> <td> 0.942</td> <td> 0.891</td> <td> 0.995</td> <td> 0.989</td> </tr> <tr> <td>F1</td> <td> 0.969</td> <td> 0.945</td> <td> 0.878</td> <td> 0.995</td> <td> 0.989</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td> <td> Precision</td> <td> 0.971</td> <td> 0.946</td> <td> 0.863 </td> <td> 0.994</td> <td> 0.988</td> </tr> <tr> <td> Recall</td> <td> 0.967</td> <td> 0.946</td> <td> 0.870</td> <td> 0.995</td> <td> 0.988</td> </tr> <tr> <td>F1</td> <td> 0.969</td> <td> 0.946</td> <td> 0.866</td> <td> 0.994</td> <td> 0.988</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td> <td> Precision</td> <td> 0.972</td> <td> 0.946</td> <td> 0.865</td> <td> 0.995</td> <td> 0.987</td> </tr> <tr> <td> Recall</td> <td> 0.970</td> <td> 0.950</td> <td> 0.905</td> <td> 0.995</td> <td> 0.987</td> </tr> <tr> <td>F1</td> <td> 0.971</td> <td> 0.948</td> <td> 0.885</td> <td> 0.995</td> <td> 0.987</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td> <td> Precision</td> <td> 0.973</td> <td> 0.953</td> <td> 0.873</td> <td> 0.996</td> <td> 0.990</td> </tr> <tr> <td> Recall</td> <td> 0.990</td> <td> 0.948</td> <td> 0.913</td> <td> 0.995</td> <td> 0.990</td> </tr> <tr> <td>F1</td> <td> 0.972</td> <td> 0.950</td> <td> 0.893</td> <td> 0.996</td> <td> 0.990</td> </tr> </tr> </tbody> </table> </details>

wikiann

For space reasons, we show only the F1 of the different models. You can see the full results below the table.

<table> <thead> <tr> <th> Model</th> <th> PER</th> <th> LOC</th> <th> ORG</th> </tr> </thead> <tbody> <tr> <td rowspan="1"> <a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td> <td> 0.867</td> <td> 0.722</td> <td> 0.451</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td> <td> 0.862</td> <td> 0.722</td> <td> 0.451</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td> <td> 0.947</td> <td> 0.906</td> <td> 0.886</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmembert2-3entities (111M)</a></td> <td> 0.950</td> <td> 0.911</td> <td> 0.910</td> </tr> <tr> <td rowspan="1"> NERmemberta-3entities (111M) (this model)</td> <td> 0.953</td> <td> 0.902</td> <td> 0.890</td> </tr> <tr> <td rowspan="1"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td> <td> 0.949</td> <td> 0.912</td> <td> 0.899</td> </tr> </tr> </tbody> </table> <details> <summary>Full results</summary> <table> <thead> <tr> <th> Model</th> <th> Metrics</th> <th> PER</th> <th> LOC</th> <th> ORG</th> <th> O</th> <th> Overall</th> </tr> </thead> <tbody> <tr> <td rowspan="3"> <a href="https://hf.co/Jean-Baptiste/camembert-ner">Jean-Baptiste/camembert-ner (110M)</a></td> <td> Precision</td> <td> 0.862</td> <td> 0.700</td> <td> 0.864</td> <td> 0.867</td> <td> 0.832</td> </tr> <tr> <td> Recall</td> <td> 0.871</td> <td> 0.746</td> <td> 0.305</td> <td> 0.950</td> <td> 0.772</td> </tr> <tr> <td>F1</td> <td> 0.867</td> <td> 0.722</td> <td> 0.451</td> <td> 0.867</td> <td> 0.801</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/cmarkea/distilcamembert-base-ner">cmarkea/distilcamembert-base-ner (67.5M)</a></td> <td> Precision</td> <td> 0.862</td> <td> 0.700</td> <td> 0.864</td> <td> 0.867</td> <td> 0.832</td> </tr> <tr> <td> Recall</td> <td> 0.871</td> <td> 0.746</td> <td> 0.305</td> <td> 0.950</td> <td> 0.772</td> </tr> <tr> <td>F1</td> <td> 0.867</td> <td> 0.722</td> <td> 0.451</td> <td> 0.907</td> <td> 0.800</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert-base-3entities">NERmembert-base-3entities (110M)</a></td> <td> Precision</td> <td> 0.948</td> <td> 0.900</td> <td> 0.893</td> <td> 0.979</td> <td> 0.942</td> </tr> <tr> <td> Recall</td> <td> 0.946</td> <td> 0.911</td> <td> 0.878</td> <td> 0.982</td> <td> 0.942</td> </tr> <tr> <td>F1</td> <td> 0.947</td> <td> 0.906</td> <td> 0.886</td> <td> 0.980</td> <td> 0.942</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert2-3entities">NERmembert2-3entities (111M)</a></td> <td> Precision</td> <td> 0.962</td> <td> 0.906</td> <td> 0.890</td> <td> 0.971</td> <td> 0.941</td> </tr> <tr> <td> Recall</td> <td> 0.938</td> <td> 0.917</td> <td> 0.884</td> <td> 0.982</td> <td> 0.941</td> </tr> <tr> <td>F1</td> <td> 0.950</td> <td> 0.911</td> <td> 0.887</td> <td> 0.976</td> <td> 0.941</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmemberta-3entities">NERmemberta-3entities (111M) (this model)</a></td> <td> Precision</td> <td> 0.961</td> <td> 0.902</td> <td> 0.899</td> <td> 0.972</td> <td> 0.942</td> </tr> <tr> <td> Recall</td> <td> 0.946</td> <td> 0.918</td> <td> 0.881</td> <td> 0.982</td> <td> 0.942</td> </tr> <tr> <td>F1</td> <td> 0.953</td> <td> 0.902</td> <td> 0.890</td> <td> 0.977</td> <td> 0.942</td> </tr> <tr> <td rowspan="3"> <a href="https://hf.co/CATIE-AQ/NERmembert-large-3entities">NERmembert-large-3entities (336M)</a></td> <td> Precision</td> <td> 0.958</td> <td> 0.917</td> <td> 0.897</td> <td> 0.980</td> <td> 0.948</td> </tr> <tr> <td> Recall</td> <td> 0.940</td> <td> 0.915</td> <td> 0.901</td> <td> 0.983</td> <td> 0.948</td> </tr> <tr> <td>F1</td> <td> 0.949</td> <td> 0.912</td> <td> 0.899</td> <td> 0.983</td> <td> 0.948</td> </tr> </tbody> </table> </details>

Usage

Code

from transformers import pipeline

ner = pipeline('token-classification', model='CATIE-AQ/NERmemberta-base-3entities', tokenizer='CATIE-AQ/NERmemberta-base-3entities', aggregation_strategy="simple")

result = ner(
"Le dévoilement du logo officiel des JO s'est déroulé le 21 octobre 2019 au Grand Rex. Ce nouvel emblème et cette nouvelle typographie ont été conçus par le designer Sylvain Boyer avec les agences Royalties & Ecobranding. Rond, il rassemble trois symboles : une médaille d'or, la flamme olympique et Marianne, symbolisée par un visage de femme mais privée de son bonnet phrygien caractéristique. La typographie dessinée fait référence à l'Art déco, mouvement artistique des années 1920, décennie pendant laquelle ont eu lieu pour la dernière fois les Jeux olympiques à Paris en 1924. Pour la première fois, ce logo sera unique pour les Jeux olympiques et les Jeux paralympiques."
)

print(result)

Try it through Space

A Space has been created to test the model. It is available here.

Environmental Impact

Carbon emissions were estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). The hardware, runtime, cloud provider, and compute region were utilized to estimate the carbon impact.

Hardware Type: A100 PCIe 40/80GB
Hours used: 2h51min
Cloud Provider: Private Infrastructure
Carbon Efficiency (kg/kWh): 0.047 (estimated from electricitymaps for the day of November 20, 2024.)
Carbon Emitted (Power consumption x Time x Carbon produced based on location of power grid): 0.0335 kg eq. CO2

Citations

NERmemBERTa-3entities

@misc {NERmemberta2024,
    author       = { {BOURDOIS, Loïck} },  
    organization = { {Centre Aquitain des Technologies de l'Information et Electroniques} },  
	title        = { NERmemberta-3entities (Revision 989f2ee) },
	year         = 2024,
	url          = { https://huggingface.co/CATIE-AQ/NERmemberta-3entities },
	doi          = { 10.57967/hf/3640 },
	publisher    = { Hugging Face }
}

NERmemBERT

@misc {NERmembert2024,
    author       = { {BOURDOIS, Loïck} },  
    organization = { {Centre Aquitain des Technologies de l'Information et Electroniques} },  
	title        = { NERmembert-base-3entities },
	year         = 2024,
	url          = { https://huggingface.co/CATIE-AQ/NERmembert-base-3entities },
	doi          = { 10.57967/hf/1752 },
	publisher    = { Hugging Face }
}

CamemBERT

@inproceedings{martin2020camembert,  
  title={CamemBERT: a Tasty French Language Model},  
  author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t},  
  booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},  
  year={2020}}

CamemBERT 2.0

@misc{antoun2024camembert20smarterfrench,
      title={CamemBERT 2.0: A Smarter French Language Model Aged to Perfection}, 
      author={Wissam Antoun and Francis Kulumba and Rian Touchent and Éric de la Clergerie and Benoît Sagot and Djamé Seddah},
      year={2024},
      eprint={2411.08868},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2411.08868}, 
}

multiconer

@inproceedings{multiconer2-report,  
    title={{SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)}},  
    author={Fetahu, Besnik and Kar, Sudipta and Chen, Zhiyu and Rokhlenko, Oleg and Malmasi, Shervin},  
    booktitle={Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)},  
    year={2023},  
    publisher={Association for Computational Linguistics}}

@article{multiconer2-data,  
    title={{MultiCoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognition}},  
    author={Fetahu, Besnik and Chen, Zhiyu and Kar, Sudipta and Rokhlenko, Oleg and Malmasi, Shervin},  
    year={2023}}

multinerd

@inproceedings{tedeschi-navigli-2022-multinerd,  
    title = "{M}ulti{NERD}: A Multilingual, Multi-Genre and Fine-Grained Dataset for Named Entity Recognition (and Disambiguation)",  
    author = "Tedeschi, Simone and  Navigli, Roberto",  
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2022",  
    month = jul,  
    year = "2022",  
    address = "Seattle, United States",  
    publisher = "Association for Computational Linguistics",  
    url = "https://aclanthology.org/2022.findings-naacl.60",  
    doi = "10.18653/v1/2022.findings-naacl.60",  
    pages = "801--812"}

pii-masking-200k

@misc {ai4privacy_2023,  
    author = { {ai4Privacy} },  
    title = { pii-masking-200k (Revision 1d4c0a1) },  
    year = 2023,  
    url = { https://huggingface.co/datasets/ai4privacy/pii-masking-200k },  
    doi = { 10.57967/hf/1532 },  
    publisher = { Hugging Face }}

wikiann

@inproceedings{rahimi-etal-2019-massively,  
    title = "Massively Multilingual Transfer for {NER}",  
    author = "Rahimi, Afshin and Li, Yuan and Cohn, Trevor",  
    booktitle = "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics",  
    month = jul,  
    year = "2019",  
    address = "Florence, Italy",  
    publisher = "Association for Computational Linguistics",  
    url = "https://www.aclweb.org/anthology/P19-1015",  
    pages = "151--164"}

wikiner

@article{NOTHMAN2013151,  
    title = {Learning multilingual named entity recognition from Wikipedia},  
    journal = {Artificial Intelligence},  
    volume = {194},  
    pages = {151-175},  
    year = {2013},  
    note = {Artificial Intelligence, Wikipedia and Semi-Structured Resources},  
    issn = {0004-3702},  
    doi = {https://doi.org/10.1016/j.artint.2012.03.006},  
    url = {https://www.sciencedirect.com/science/article/pii/S0004370212000276},  
    author = {Joel Nothman and Nicky Ringland and Will Radford and Tara Murphy and James R. Curran}}

frenchNER_3entities

@misc {frenchNER2024,  
    author       = { {BOURDOIS, Loïck} },  
    organization  = { {Centre Aquitain des Technologies de l'Information et Electroniques} },  
    title        = { frenchNER_3entities },  
    year         = 2024,  
    url          = { https://huggingface.co/CATIE-AQ/frenchNER_3entities },  
    doi          = { 10.57967/hf/1751 },  
    publisher    = { Hugging Face }  
}

License

MIT

Task: Token Classification
Library: transformers
Type: deberta-v2
License: mit
Languages: fr
Created: Nov 20, 2024
Updated: Dec 5, 2024