Mango-MT-9B

Mango-MT: Advancing Audiovisual Translation with a 9B Multilingual Model and Expert-Annotated Benchmark

<img width="6303" height="1881" alt="f0" src="https://github.com/user-attachments/assets/68f72a09-0d9b-4e95-b9e6-f2a5a4d420e4" />

Introduction

Global long-form video expansion requires robust multilingual subtitle translation, but generic MT fails at fragmented lines, timeline constraints and plot context for mass production. We open-sourced Mango-MT with 9B parameter size, an 11-language audiovisual subtitle translator paired with the benchmark called Mango-SubBench. Evaluations across all languages show it outperforms GPT, Gemini and DeepSeek with steady timeline compliance, consistent semantics and industrial scalability, see the technical report and GitHub for details. Our system holds four key advantages over prior work:

• Scenario-oriented & Multilingual Optimization: Professionally optimized for 11 different languages, perfectly adapting to complex industrial rules of video subtitle translation.
• Context-aware Translation Mechanism: Leverages global context modeling to stabilize plot logic and consistent character appellation translation.
• Structural Robustness: Rigidly retains original subtitle numbers and timestamps to ensure stable batch translation delivery.
• Industrial-grade evaluation benchmark : Unlike single-metric schemes, our benchmark integrates a full framework and high-quality dataset, evaluating subtitles against real delivery standards for reliable all-round quality assessment.

Installation

conda create -n mlt python==3.11
conda activate mlt
pip install -r requirements.txt

Usage

SGLang Server Usage

SGLang is a fast serving framework for large language models and vision language models. Please use the following command in a fresh environment:

uv pip install 'git+https://github.com/sgl-project/sglang.git#subdirectory=python&egg=sglang[all]'

See its documentation for more details.

The following will create API endpoints at http://localhost:8000/v1:

python -m sglang.launch_server --model-path model_dir --port 8000 --tp-size 1 --mem-fraction-static 0.8 --context-length 4096 --reasoning-parser qwen3

vLLM Server Usage

vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs. Please use the following command in a fresh environment:

uv pip install vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly

See its documentation for more details.

For detailed usage guide, see the vLLM Qwen3.5 recipe.

The following will create API endpoints at http://localhost:8000/v1:

vllm serve model_dir --port 8000 --tensor-parallel-size 1 --max-model-len 4096 --reasoning-parser qwen3 --language-model-only

Performance

Mango-SubBench

Mango-SubBench is a novel benchmark for audiovisual subtitle translation. We evaluate our Mango-MT against three commercial large models (DeepSeek-v4-pro, Gemini-3-pro, GPT-5.4) across 11 languages with five core translation metrics: Semantic similarity, BLEU-2, COMET On translation benchmark, Mango-MT outperforms Gemini, DeepSeek and GPT across all 11 evaluated languages. This demonstrates that our model delivers powerful multilingual translation capabilities for film and drama content, and possesses significant commercial deployment value.

Evaluation Results on S(overall score) are:

FLORES+

We evaluate the multilingual translation performance of our model on FLORES+ which is based on FLORES-200. This dataset was originally released by FAIR researchers at Meta under the name FLORES. The data is now being managed by OLDI, the Open Language Data Initiative. The + has been added to the name to disambiguate between the original datasets and this new actively developed version. For newer versions of this dataset, Please see FLORES+ HuggingFace repo . The data consists of translations primarily from English into over 200 language varieties. The original English sentences were sampled in equal amounts from Wikinews (an international news source), Wikijunior (a collection of age-appropriate non-fiction books), and Wikivoyage (a travel guide).