Downloads · 30 days
91
80% of all-time downloads
adventists-ai-research/DuplexJev-A-Qwen3-ASR-0.6B-Qwen3-32B
DuplexJev-A-Qwen3-ASR-0.6B-Qwen3-32B is a audio-text-to-text model from adventists-ai-research. Use it for the audio-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
DuplexJev reads typed, closed-set decisions from speech (is the turn over? which filler fits? who is speaking?) as a single-token distribution of a frozen LLM, with zero decode steps and many questions per forward pas…
Downloads · 30 days
91
80% of all-time downloads
All-time downloads
114
Public
Parameters
21.1M
53.5 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors42.1 MB · 72%
From the Hugging Face model README
DuplexJev reads typed, closed-set decisions from speech (is the turn over? which filler fits? who is speaking?) as a single-token distribution of a frozen LLM, with zero decode steps and many questions per forward pass. This repository holds one connector that joins a frozen ASR encoder to a frozen LLM, in Ultravox format.
| ASR encoder (frozen) | adventists-ai/Qwen3-ASR-0.6B-Encoder-XAttn (from Qwen/Qwen3-ASR-0.6B) |
| LLM (frozen) | Qwen/Qwen3-32B |
| Connector | A (cross-attention fusion): a fusion block (queries from encoder layer 18, keys from layer 14, values from layer 9; zero-initialised, added residually) followed by the projector. 21.1 M trainable parameters (fusion 3.2 M + projector 17.8 M). |
| Audio tokens | 6.25 per second (stack factor 2) |
| This variant | Round 2 of content alignment (transcription + continuation, transcript distillation). No paralinguistic training: gender and emotion are at chance. Use it as the starting point for your own decision fine-tuning. |
pip install "duplexjev[speech]>=0.2.1"
from duplexjev import Decider, Question
d = Decider.from_pretrained("adventists-ai/DuplexJev-A-Qwen3-ASR-0.6B-Qwen3-32B", device="auto") # Qwen3-32B bf16: ~65 GB of GPU memory
# English clip
d.decide("utterance_en.wav", [
Question("turn", "Has the user finished speaking?", ["finished", "not finished"], lang="en"),
Question("intent", "What does the user want?", ["climate", "media", "navigation", "phone", "chit-chat"], lang="en"),
], lang="en")
# Chinese clip: ask in Chinese, with Chinese options
d.decide("utterance_zh.wav", [
Question("turn", "用户说完了吗?", ["说完了", "还没说完"], lang="zh"),
Question("intent", "用户想做什么?", ["空调", "媒体", "导航", "电话", "闲聊"], lang="zh"),
], lang="zh")
# -> {question_id: {"answer": ..., "confidence": ..., "probs": {option: p, ...}}}
Ask in the language of the clip, with options worded as in the examples: the connector was trained with
language-matched prompts. Use duplexjev>=0.2.1; 0.2.0 padded audio at the start, which costs several points.
Already have local copies of the encoder and LLM? Pass text_model="/path/Qwen3-32B" and
audio_model="/path/Qwen3-ASR-0.6B-Encoder-XAttn"; this works offline.
decide_batch answers question groups over many clips in one pass. See the GitHub repository for the
batched pipeline, prefix sharing, the serving loop and evaluation scripts.
| Evaluation | Score (%) |
|---|---|
| qa100 (spoken QA, /100) | 83 |
| ZJU-ML (spoken QA, /100) | 77 |
| Easy-Turn (zero-shot turn state) | 77.1 |
| Gender, 800 real utterances | 55 (chance) |
| Emotion, 4-way, 800 utterances | 28 (chance) |
qa100: 100 bilingual spoken multiple-choice questions (adventists-ai/qa100).
ZJU-ML: main-language part of the ZJU audio benchmark v2.0.0 (50 real-speech factual + 50 TTS math/logic items).
Numbers are single-token readouts with the paper's evaluation prompts.
Reproduced with duplexjev 0.2.1 and its default prompts (one option order, lang matched to the clip):
| Evaluation | Score (%) |
|---|---|
| Gender, 800 real utterances | 53.1 |
| Emotion, 4-way, 800 utterances | 27.0 |
| qa100 | 85 |
Apache-2.0 for the connector weights and code in this repository. The frozen base models keep their own licences (Qwen3-32B and Qwen3-ASR-0.6B: Apache-2.0). The training corpora have their own terms, and some of them (e.g. WenetSpeech, CoVoST 2) are licensed for non-commercial use only; check them before commercial use.
@inproceedings{jin2027duplexjev,
title = {Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen {LLM} Hear Beyond the Transcript},
author = {Jin, Jie and Ma, Ziyin and Yin, Min and Chen, Jinyu and Song, Haigang and Pang, Zhikun and Zhang, Xiaowen},
booktitle = {Submitted to IEEE ICASSP},
year = {2027}
}
Built by Adventists.ai. Claude (Anthropic) assisted with code.