SpeechLMM-v2.0-XL-30B

meetween/SpeechLMM-v2.0-XL-30B

SpeechLMM-v2.0-XL-30B is a feature extraction model from meetween. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as cc-by-nc-sa-4.0.

Downloads · 30 days

8% of all-time downloads

All-time downloads

376

Public

Parameters

35.4B

207 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors70.9 GB · 100%

At a glance

Task: Feature Extraction
Library: transformers
License: cc-by-nc-sa-4.0
Model type: speechlmm
Access: Public
Created: Jul 20, 2026
Updated: Sep 30, 2026
SHA: 0214e9c4

Base models

Qwen/Qwen3-Omni-30B-A3B-Instruct

Model card

From the Hugging Face model README

Model information (WIP)

The SpeechLMM 2.0 collection of multimodal and multilingual large language models is a collection of instruction-tuned generative models in 4 different sizes: S (1.7B), M (3B), L (7B) and XL (30B), supporting text, audio and video as input and text and audio as output. The SpeechLMM 2.0 models are optimized for various X-to-X generation tasks, namely:

Audio Chaptering (ACHAP)
Automatic Speech Recognition (ASR)
Audiovisual Speaker Diarization (AVSPEAKD)
Lip Reading (LIPREAD)
Machine Translation (MT)
Speech-to-speech Translation (S2ST)
Spoken Language Understanding - Intent (SLU-I)
Spoken Question Answering - Abstractive (SQA-A)
Spoken Question Answering - Extractive (SQA-E)
Spoken Question Answering - Multiple Choice (SQA-M)
Speech Summarization (SSUM)
Speech Translation (ST)
Textual Question Answering - Abstractive (TQA-A)
Textual Question Answering - Extractive (TQA-E)
Textual Question Answering - Multiple Choice (TQA-M)
Text Summarization (TSUM)
Text-to-Speech (TTS)

Model Developer: Meetween consortium

Supported Languages: Czech, German, English, Spanish, French, Hungarian, Italian, Dutch, Portuguese, Swedish are officially supported (for a subset of the supported tasks). The Qwen Omni backbone have been originally trained on a broader collection of languages than these 10 supported ones, therefore, the model might exhibit good performance on other languages too.

Model Release Date: July 31, 2026

License: Apache 2.0

Model Architecture (WIP)

SpeechLMM 2.0 is an auto-regressive multimodal language model based on a Qwen-X-Omni backbone (X varies with the model size) and incorporates a dedicated visual speech recognition pathway.

The architecture consists of:

Qwen3-Omni-30B-A3B-Instruct as the multimodal foundation model backbone;
The Qwen3 Thinker module for multimodal reasoning;
The Qwen3 Talker module for speech generation;
The Audio Transformer (AuT) encoder inherited from Qwen3-Omni for speech processing;
A SigLIP2-So400M-based [2] vision encoder;
An AutoAVSR-based [3] lip-reading encoder for extracting visual speech representations;
Trainable adapter layers connecting video, lip-reading and audio encoders outputs to the Qwen3 Thinker representation space.

Model	Backbone	Params	Input modalities	Output modalities	Context Length
SpeechLMM 2.0 S	---	1.7B	Multilingual text and audio, English video	Multilingual Text	---
SpeechLMM 2.0 M	Qwen-2.5-Omni-3B	3B	Multilingual text and audio, English video	Multilingual Text	---
SpeechLMM 2.0 L	Qwen-2.5-Omni-7B	7B	Multilingual text and audio, English video	Multilingual Text	---
SpeechLMM 2.0 XL	Qwen-3.0-Omni-30B-A3B-Instruct	30B	Multilingual text and audio, English video	Multilingual Text	---

Training Details (WIP)

How to use (WIP)

Refer to the instructions in our codebase:

Training Data (WIP)

TASK	Dataset	Language	License
ACHAP	AMI Meeting Corpus	en → en	-
	YTSeg	en → en	-
ASR	FLEURS	cs → cs, de → de, en → en, es → es, fr → fr, hu → hu, it → it, nl → nl, pt → pt, sv → sv	-
	Multilingual Spoken Topical-Chat	cs → cs, de → de, en → en, es → es, fr → fr, hu → hu, it → it, nl → nl, pt → pt, sv → sv	-
	Spoken DGT-TM	cs → cs, de → de, en → en, es → es, fr → fr, hu → hu, it → it, nl → nl, pt → pt, sv → sv	-
	VoxPopuli	cs → cs, de → de, en → en, es → es, fr → fr, hu → hu, it → it, nl → nl, pt → pt, sv → sv	-
	EuroSpeech	cs → cs, de → de, en → en, es → es, fr → fr, hu → hu, it → it, nl → nl, pt → pt, sv → sv	-
AVSPEAKD	AMI Meeting Corpus	en → en	-
LIPREAD	LipCrops	en → en	-
MT	Europarl-ST	{de, en, es, fr, it, nl, pt} → {de, en, es, fr, it, nl, pt}	-
	Multilingual Spoken Topical-Chat	{cs, de, en, es, fr, hu, it, nl, pt, sv} → {cs, de, en, es, fr, hu, it, nl, pt, sv}	-
	Spoken DGT-TM	{cs, de, en, es, fr, hu, it, nl, pt, sv} → {cs, de, en, es, fr, hu, it, nl, pt, sv}	-
SLU-I	SLURP	en → en	-
	Speech-MASSIVE	de → de, fr → fr	-
SQA-E	Multilingual Spoken SQuAD	cs → cs, de → de, en → en, es → es, fr → fr, hu → hu, it → it, nl → nl, pt → pt, sv → sv	-
SSUM	ICSI	en → en	-
ST	Europarl-ST	{de, en, es, fr, it, nl, pt} → {de, en, es, fr, it, nl, pt}	-
	Multilingual Spoken Topical-Chat	{cs, de, en, es, fr, hu, it, nl, pt, sv} → {cs, de, en, es, fr, hu, it, nl, pt, sv}	-
	Spoken DGT-TM	{cs, de, en, es, fr, hu, it, nl, pt, sv} → {cs, de, en, es, fr, hu, it, nl, pt, sv}	-
TQA-E	Multilingual Spoken SQuAD	cs → cs, de → de, en → en, es → es, fr → fr, hu → hu, it → it, nl → nl, pt → pt, sv → sv	-
TQA-M	Multilingual Text LibriSQA	cs → cs, de → de, en → en, es → es, fr → fr, hu → hu, it → it, nl → nl, pt → pt, sv → sv	-
TSUM	AMI Meeting Corpus	en → en	-
	ELITR Minuting Corpus	cs → cs, en → en	-

Evaluation Results (WIP)

The following results specifically refer to the L model.

ASR Metrics (WIP)

SLU Metrics (WIP)

SQA Metrics (WIP)

SSUM Metrics (WIP)

ST Metrics (WIP)

LIPREAD Metrics (WIP)

MT Metrics (WIP)

TSUM Metrics (WIP)

Framework versions (WIP)

Task: Feature Extraction
Library: transformers
Type: speechlmm
License: cc-by-nc-sa-4.0
Created: Jul 20, 2026
Updated: Sep 30, 2026