Llama-speechlmm-1.0-l-MT

Model Information

This is the version of meetween/Llama-speechlmm-1.0-l that was fine-tuned for Speech-to-Text Translation.

License: see LICENSE

Model Architecture

Identical to the base model. The model was obtained by training LoRA on the LLM. This repository contains the model weights with LoRA merged into the main weights.

How to Use

Identical to the base model.

Fine-tuning Data

This model has been fine-tuned on the same EuroParl-ST machine translation data ({en, fr, it, de, es} → {en, fr, it, de, es}) from the training data of the base model.

Evaluation Results

<style type="text/css"> .tg {border-collapse:collapse;border-spacing:0;} .tg td{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px; overflow:hidden;padding:10px 5px;word-break:normal;} .tg th{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px; font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;} .tg .tg-8bgf{border-color:inherit;font-style:italic;text-align:center;vertical-align:top} .tg .tg-7btt{border-color:inherit;font-weight:bold;text-align:center;vertical-align:top} .tg .tg-0pky{border-color:inherit;text-align:left;vertical-align:top} .tg .tg-fymr{border-color:inherit;font-weight:bold;text-align:left;vertical-align:top} </style> <table> <colgroup> <col width="137"> <col width="52"> <col width="60"> <col width="61"> <col width="56"> <col width="58"> <col width="58"> <col width="59"> </colgroup> <tbody> <tr> <td> <p dir="ltr"><span>DATASET:</span></p> </td> <td colspan="4"> <p dir="ltr"><span>FLORES</span></p> </td> <td colspan="2"> <p dir="ltr"><span>ACL 60/60</span></p> </td> <td rowspan="2"> <p dir="ltr"><span>AVG</span></p> </td> </tr> <tr> <td> <p dir="ltr"><span>BLEU</span></p> </td> <td> <p dir="ltr"><span>en-de</span></p> </td> <td> <p dir="ltr"><span>en-es</span></p> </td> <td> <p dir="ltr"><span>en-it</span></p> </td> <td> <p dir="ltr"><span>en-fr</span></p> </td> <td> <p dir="ltr"><span>en-fr</span></p> </td> <td> <p dir="ltr"><span>en-de</span></p> </td> </tr> <tr> <td> <p dir="ltr"><span>Llama3-instruct (D5)</span></p> </td> <td> <p dir="ltr"><span>28.1</span></p> </td> <td> <p dir="ltr"><span>24.4</span></p> </td> <td> <p dir="ltr"><span>25.0</span></p> </td> <td> <p dir="ltr"><span>41.2</span></p> </td> <td> <p dir="ltr"><span>48.8</span></p> </td> <td> <p dir="ltr"><span>34.2</span></p> </td> <td> <p dir="ltr"><span>33.6</span></p> </td> </tr> <tr> <td> <p dir="ltr"><span>NLLB (D5)</span></p> </td> <td> <p dir="ltr"><span>39.4</span></p> </td> <td> <p dir="ltr"><span>23.7</span></p> </td> <td> <p dir="ltr"><span>31.2</span></p> </td> <td> <p dir="ltr"><span>50.7</span></p> </td> <td> <p dir="ltr"><span>59.1</span></p> </td> <td> <p dir="ltr"><span>45.2</span></p> </td> <td> <p dir="ltr"><span>41.6</span></p> </td> </tr> <tr> <td> <p dir="ltr"><span>SpeechLMM_v1.0_L</span></p> </td> <td> <p dir="ltr"><span>29.4</span></p> </td> <td> <p dir="ltr"><span>22.3</span></p> </td> <td> <p dir="ltr"><span>20.1</span></p> </td> <td> <p dir="ltr"><span>31.9</span></p> </td> <td> <p dir="ltr"><span>35.5</span></p> </td> <td> <p dir="ltr"><span>32.8</span></p> </td> <td> <p dir="ltr"><span>28.7</span></p> </td> </tr> <tr> <td> <p dir="ltr"><span>Speech LMM v1.0_L-FT (LoRA)</span></p> </td> <td> <p dir="ltr"><span>20.0</span></p> </td> <td> <p dir="ltr"><span>16.0</span></p> </td> <td> <p dir="ltr"><span>11.6</span></p> </td> <td> <p dir="ltr"><span>21.8</span></p> </td> <td> <p dir="ltr"><span>24.9</span></p> </td> <td> <p dir="ltr"><span>20.7</span></p> </td> <td> <p dir="ltr"><span>19.2</span></p> </td> </tr> </tbody> </table>

Framework Versions

• Transformers 4.45.0
• Pytorch 2.3.1+cu124.post2
• Datasets 3.2.0
• Tokenizers 0.20.0