nordic-roberta-wiki

flax-community/nordic-roberta-wiki

nordic-roberta-wiki is a fill-mask model from flax-community. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as cc-by-4.0.

Nord roberta model trainined on the swedish danish and norwegian wikipedia.

Downloads · 30 days

1% of all-time downloads

All-time downloads

2.9K

Public

Repo size

2.5 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.msgpack499 MB · 50%

At a glance

Task: Fill-Mask
Library: transformers
License: cc-by-4.0
Model type: roberta
Access: Public
Created: Mar 2, 2022
Updated: Sep 23, 2021
SHA: 9f040084

Try a prompt

Model card

From the Hugging Face model README

Nordic Roberta Wikipedia

Description

Nord roberta model trainined on the swedish danish and norwegian wikipedia.

Evaluation

Evaluation on Named Entity recognition in Danish.

I finetuned each model on 3 epochs on DaNE, repeated it 5 times for each model, and calculated 95% confidence intervals for the means. Here are the results:

xlm-roberta-base : 88.01 +- 0.43

flax-community/nordic-roberta-wiki: 85.75 +- 0.69 (this model)

Maltehb/danish-bert-botxo: 85.38 +- 0.55

flax-community/roberta-base-danish: 80.14 +- 1.47

flax-community/roberta-base-scandinavian : 78.03 +- 3.02

Maltehb/-l-ctra-danish-electra-small-cased: 57.87 +- 3.19

NbAiLab/nb-bert-base : 30.24 +- 1.21

Randomly initialised RoBERTa model: 19.79 +- 2.00

Evaluation on Sentiment analysis in Dansish Here are the results on test set, where each model has been trained 5 times, and the “+-” refers to a 95% confidence interval of the mean score:

Maltehb/danish-bert-botxo: 65.19 +- 0.53

NbAiLab/nb-bert-base : 63.80 +- 0.77

xlm-roberta-base : 63.55 +- 1.59

flax-community/nordic-roberta-wiki : 56.46 +- 1.77

flax-community/roberta-base-danish : 54.73 +- 8.96

flax-community/roberta-base-scandinavian : 44.28 +- 9.21

Maltehb/-l-ctra-danish-electra-small-cased : 47.78 +- 12.65

Randomly initialised RoBERTa model: 36.96 +- 1.02

Maltehb/roberta-base-scandinavian : 33.65 +- 8.32

Model series

This model is part of a series of models training on TPU with Flax Jax during Huggingface Flax/Jax challenge.

Gpt models

Roberta models

Nordic Roberta Wiki

https://huggingface.co/flax-community/nordic-roberta-wiki

Swe Roberta Wiki Oscar

https://huggingface.co/flax-community/swe-roberta-wiki-oscar

Roberta Swedish Scandi

https://huggingface.co/birgermoell/roberta-swedish-scandi

Roberta Swedish

https://huggingface.co/birgermoell/roberta-swedish

Swedish T5 model

https://huggingface.co/birgermoell/t5-base-swedish

Task: Fill-Mask
Library: transformers
Type: roberta
License: cc-by-4.0
Languages: sv
Created: Mar 2, 2022
Updated: Sep 23, 2021