Downloads · 30 days
103
77% of all-time downloads
adventists-ai-research/DuplexJev-B-Qwen3-ASR-0.6B-Qwen3-32B
DuplexJev-B-Qwen3-ASR-0.6B-Qwen3-32B is a audio-text-to-text model from adventists-ai-research. Use it for the audio-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
DuplexJev reads typed, closed-set decisions from speech (is the turn over? which filler fits? who is speaking?) as a single-token distribution of a frozen LLM, with zero decode steps and many questions per forward pas…
Downloads · 30 days
103
77% of all-time downloads
All-time downloads
133
Public
Parameters
17.8M
47.1 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors35.7 MB · 69%
From the Hugging Face model README
DuplexJev reads typed, closed-set decisions from speech (is the turn over? which filler fits? who is speaking?) as a single-token distribution of a frozen LLM, with zero decode steps and many questions per forward pass. This repository holds one connector that joins a frozen ASR encoder to a frozen LLM, in Ultravox format.
| ASR encoder (frozen) | adventists-ai/Qwen3-ASR-0.6B-Encoder (from Qwen/Qwen3-ASR-0.6B) |
| LLM (frozen) | Qwen/Qwen3-32B |
| Connector | B (last layer): the final encoder layer (h18) goes straight into the projector (frame stacking ×2 + SwiGLU MLP). 17.8 M trainable parameters. |
| Audio tokens | 6.25 per second (stack factor 2) |
| This variant | Round 2 of content alignment (transcription + continuation, transcript distillation). The best content connector in the paper. No paralinguistic training. |
pip install "duplexjev[speech]>=0.2.1"
from duplexjev import Decider, Question
d = Decider.from_pretrained("adventists-ai/DuplexJev-B-Qwen3-ASR-0.6B-Qwen3-32B", device="auto") # Qwen3-32B bf16: ~65 GB of GPU memory
# English clip
d.decide("utterance_en.wav", [
Question("turn", "Has the user finished speaking?", ["finished", "not finished"], lang="en"),
Question("intent", "What does the user want?", ["climate", "media", "navigation", "phone", "chit-chat"], lang="en"),
], lang="en")
# Chinese clip: ask in Chinese, with Chinese options
d.decide("utterance_zh.wav", [
Question("turn", "用户说完了吗?", ["说完了", "还没说完"], lang="zh"),
Question("intent", "用户想做什么?", ["空调", "媒体", "导航", "电话", "闲聊"], lang="zh"),
], lang="zh")
# -> {question_id: {"answer": ..., "confidence": ..., "probs": {option: p, ...}}}
Ask in the language of the clip, with options worded as in the examples: the connector was trained with
language-matched prompts. Use duplexjev>=0.2.1; 0.2.0 padded audio at the start, which costs several points.
Already have local copies of the encoder and LLM? Pass text_model="/path/Qwen3-32B" and
audio_model="/path/Qwen3-ASR-0.6B-Encoder"; this works offline.
decide_batch answers question groups over many clips in one pass. See the GitHub repository for the
batched pipeline, prefix sharing, the serving loop and evaluation scripts.
| Evaluation | Score (%) |
|---|---|
| qa100 (spoken QA, /100) | 90 (oracle transcript: 91) |
| ZJU-ML (spoken QA, /100) | 79 |
| Easy-Turn (zero-shot turn state) | 76.1 |
| Gender, 800 real utterances | 54 (chance) |
| Emotion, 4-way, 800 utterances | 27 (chance) |
qa100: 100 bilingual spoken multiple-choice questions (adventists-ai/qa100).
ZJU-ML: main-language part of the ZJU audio benchmark v2.0.0 (50 real-speech factual + 50 TTS math/logic items).
Numbers are single-token readouts with the paper's evaluation prompts.
Reproduced with duplexjev 0.2.1 and its default prompts (one option order, lang matched to the clip):
| Evaluation | Score (%) |
|---|---|
| Gender, 800 real utterances | 52.5 |
| Emotion, 4-way, 800 utterances | 26.6 |
| qa100 | 88 |
Apache-2.0 for the connector weights and code in this repository. The frozen base models keep their own licences (Qwen3-32B and Qwen3-ASR-0.6B: Apache-2.0). The training corpora have their own terms, and some of them (e.g. WenetSpeech, CoVoST 2) are licensed for non-commercial use only; check them before commercial use.
@inproceedings{jin2027duplexjev,
title = {Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen {LLM} Hear Beyond the Transcript},
author = {Jin, Jie and Ma, Ziyin and Yin, Min and Chen, Jinyu and Song, Haigang and Pang, Zhikun and Zhang, Xiaowen},
booktitle = {Submitted to IEEE ICASSP},
year = {2027}
}
Built by Adventists.ai. Claude (Anthropic) assisted with code.