Downloads · 30 days
7
5% of all-time downloads
sentinel-ref-da
sapienzanlp/sentinel-ref-da
sentinel-ref-da is a translation model from sapienzanlp. Use it when you need text moved from one language to another. It is set up for transformers. The card lists the license as cc-by-nc-sa-4.0.
All-time downloads
149
Public
Repo size
2.2 GB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.ckpt2.2 GB · 100%
At a glance
- Task
- Translation
- Library
- transformers
- License
- cc-by-nc-sa-4.0
- Model type
- SENTINEL-REF-DA
- Access
- Public
- Created
- Aug 1, 2024
- Updated
- Aug 30, 2024
- SHA
- 550a4338
Model card
From the Hugging Face model README
<div align="center">
<h1 style="font-family: 'Arial', sans-serif; font-size: 28px; font-weight: bold; color: black;">
🛡️ Guardians of the Machine Translation Meta-Evaluation:<br>
Sentinel Metrics Fall In!
</h1>
</div>
<div style="display:flex; justify-content: center; align-items: center; flex-direction: row;">
<a href="https://2024.aclweb.org/"><img src="http://img.shields.io/badge/ACL-2024-4b44ce.svg"></a>
<a href="https://aclanthology.org/2024.acl-long.856/"><img src="http://img.shields.io/badge/paper-ACL--anthology-B31B1B.svg"></a>
<a href="https://arxiv.org/abs/2408.13831"><img src="https://img.shields.io/badge/arXiv-2408.13831-b31b1b.svg"></a>
<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/"><img src="https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey.svg"></a>
</div>
<div style="display:flex; justify-content: center; align-items: center; flex-direction: row;">
<a href="https://huggingface.co/collections/sapienzanlp/mt-sentinel-metrics-66ab643b32aab06f3157e5c1"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Collection-FCD21D"></a>
<a href="https://github.com/SapienzaNLP/guardians-mt-eval"><img src="https://img.shields.io/badge/GitHub-Repo-121013?logo=github&logoColor=white"></a>
</div>
This repository contains the SENTINEL<sub>REF</sub> metric model trained on Direct Assessments (DA) annotations. For details on how to use our sentinel metric models, check our GitHub repository.
Usage
After having installed our repository package, you can use this model within Python in the following way:
from sentinel_metric import download_model, load_from_checkpoint
model_path = download_model("sapienzanlp/sentinel-ref-da")
model = load_from_checkpoint(model_path)
data = [
{"ref": "There's no place like home."},
{"ref": "Toto, I've a feeling we're not in Kansas anymore."}
]
output = model.predict(data, batch_size=8, gpus=1)
Output:
# Segment scores
>>> output.scores
[0.26086926460266113, 0.17275342345237732]
# System score
>>> output.system_score
0.21681134402751923
Cite this work
This work has been published at ACL 2024 (Main Conference). If you use any part, please consider citing our paper as follows:
@inproceedings{perrella-etal-2024-guardians,
title = "Guardians of the Machine Translation Meta-Evaluation: Sentinel Metrics Fall In!",
author = "Perrella, Stefano and Proietti, Lorenzo and Scir{\`e}, Alessandro and Barba, Edoardo and Navigli, Roberto",
editor = "Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek",
booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = aug,
year = "2024",
address = "Bangkok, Thailand", publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.acl-long.856",
pages = "16216--16244",
}
License
This work is licensed under Creative Commons Attribution-ShareAlike-NonCommercial 4.0.
- Task
- Translation
- Library
- transformers
- Type
- SENTINEL-REF-DA
- License
- cc-by-nc-sa-4.0
- Languages
- multilingual
- Created
- Aug 1, 2024
- Updated
- Aug 30, 2024