A.X-K2-ALM

A.X K2 ALM — Audio Language Model

<div align="center"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/skt/A.X-K2-ALM/resolve/main/assets/A.X_K2_BI_Negative.png"> <img src="https://huggingface.co/skt/A.X-K2-ALM/resolve/main/assets/A.X_K2_BI_Primary.png" alt="A.X K2 ALM" width="300"> </picture> </div> <p align="center"> <a href="https://huggingface.co/skt/A.X-K2-ALM">🤗 Models</a> | <a href="https://huggingface.co/skt/A.X-K2-ALM/resolve/main/assets/A.X-K2-ALM-Technical-Report.pdf">📄 Technical Report</a> </p>

Model Summary

A.X K2 ALM (Audio Language Model) is a member of the A.X K2 family that adds a speech modality to a frozen in-house LLM backbone — A.X K2 Light (20B-A2.7B MoE) — to enable natural voice-based conversation. It integrates speech understanding, voice activity detection (VAD), and speech generation within a single unified, streaming architecture built from four components: a chunk-wise Conformer speech encoder, an adapter (MLP projector + Q-Former), a lightweight VAD module, and a multi-token-prediction (MTP) speech decoder.

Motivated by production settings where the deployed LLM may be audio-unaware and its existing text-response quality must be preserved, we keep the LLM entirely frozen and train only the speech-side components — avoiding the catastrophic forgetting and text-quality degradation commonly induced by end-to-end multimodal training, and letting the same recipe transfer readily to other LLMs. On our Korean KVoiceBench, A.X K2 ALM reaches 95% of Qwen3-Omni-30B-A3B's response quality at roughly two-thirds the parameters.

<div align="center"> <img src="https://huggingface.co/skt/A.X-K2-ALM/resolve/main/assets/fig_architecture.png" alt="A.X K2 ALM overall architecture" width="520"> <br/> _{<b>A.X K2 ALM architecture.</b> Speech encoder → adapter → frozen LLM → speech decoder; the VAD reads the adapter and LLM representations.} </div>

Key Features

• A Korean ALM Built Entirely from Scratch Every speech-side component — the streaming speech encoder, the adapter, the VAD heads, and the speech decoder with its neural audio codec — is built and trained in-house from scratch rather than adapted from open-source models, on top of the frozen A.X K2 Light backbone. This lets the frame rate, streaming behavior, and language coverage be tuned end-to-end for Korean voice services.

• Self-Supervised (SSL) Streaming Speech Encoder A 300M chunk-wise Conformer encoder, self-supervised pre-trained from scratch on 400K hours of unlabeled speech with multi-codebook Random Quantization and fine-tuned as a 341M streaming transducer, outperforms Qwen3-Omni on Korean ASR benchmarks at roughly half its audio-encoder size — even under a low-latency streaming constraint.

• Efficient Streaming Speech Decoder We introduce a 0.88B multi-token speech decoder that integrates a text–speech language model with a depth-transformer to hierarchically generate semantic tokens and residual acoustic codes in real time. Operating at roughly half the parameter footprint of current SOTA baselines, our model achieves superior pronunciation accuracy with the lowest CER while maintaining high speech intelligibility comparable to leading systems, ensuring precise delivery of generated text responses in real-time ALMs.

• Forgetting-Free Modality Extension The LLM stays fully frozen; a two-stage adapter — speech alignment (syllable-CTC / char2token bridge) then self-distillation dialogue training — transfers the backbone's response behavior to the audio path without changing its text-input behavior.

• Integrated Context-Aware VAD The external VAD is replaced by lightweight six-class heads on the model's own representations — an adapter-based head detects onset, an LLM-based head verifies with context — enabling barge-in and far fewer false triggers on non-speech and mid-utterance pauses.

Model Details

<table> <thead> <tr><th>Function</th><th>Component</th><th>Configuration</th><th>Params</th></tr> </thead> <tbody> <tr><td rowspan="5">Speech Understanding</td><td>Speech Encoder</td><td>Chunk-wise Conformer</td><td>300M</td></tr> <tr><td>Adapter — MLP Projector</td><td>2-layer, GELU</td><td>5.77M</td></tr> <tr><td>Adapter — char-CTC head</td><td>linear, 2048 → 2667</td><td>5.46M</td></tr> <tr><td>Adapter — Q-Former</td><td>4L / 16H / d=2048 / FFN 8192</td><td>201.4M</td></tr> <tr><td>LLM (frozen)</td><td>A.X K2 Light (in-house MoE)</td><td>20B-A2.7B</td></tr> <tr><td rowspan="5">Speech Generation</td><td>Conditioning Projector</td><td>FFN stack (2048 → 1024, 2 paths)</td><td>71.3M</td></tr> <tr><td rowspan="2">Multi-Token Predictor</td><td>Text–Speech Language Model</td><td>630M</td></tr> <tr><td>Depth Transformer</td><td>110M</td></tr> <tr><td>Speaker Encoder</td><td>Transformer</td><td>100M</td></tr> <tr><td>Audio Codec Decoder</td><td>Transformer + ConvNet</td><td>40M</td></tr> <tr><td><b>Total</b></td><td><b>A.X K2 ALM</b></td><td></td><td><b>≈22B-A4B</b></td></tr> </tbody> </table>

• VAD heads: six-class classifiers over the adapter output and LLM hidden states; parameter counts are negligible.
• Languages: Korean
• Streaming: all input/output paths are designed for streaming operation for low-latency live voice interaction.
• Full details: see the technical report for the architecture and training recipe.

Evaluation Results

Speech understanding is evaluated on KVoiceBench, our own Korean spoken-query benchmark suite built in-house by localizing VoiceBench with human review and adding Korean-native professional-knowledge items, covering speech recognition (STT), spoken QA (Speech-QA), and instruction following (Speech-IF). Open-ended (OPEN) categories are scored by an LLM judge and knowledge (QA) categories by correctness. See the technical report for KVoiceBench construction details. Bold = best among compared models.

<table> <thead> <tr><th>Task</th><th>Benchmark</th><th>Metric</th><th align="right">A.X K2 ALM (22B-A4B)</th><th align="right">Qwen2.5-Omni (7B)</th><th align="right">Qwen3-Omni (30B-A3B)</th><th align="right">HyperCLOVA X 8B Omni</th></tr> </thead> <tbody> <tr><td rowspan="2">STT</td><td>KsponSpeech (eval-clean)</td><td>CER ↓</td><td align="right">9.00</td><td align="right">18.96</td><td align="right"><b>8.46</b></td><td align="right">10.22</td></tr> <tr><td>KsponSpeech (eval-other)</td><td>CER ↓</td><td align="right">9.12</td><td align="right">22.72</td><td align="right"><b>7.91</b></td><td align="right">10.15</td></tr> <tr><td rowspan="9">Speech-QA</td><td>AdvBench</td><td>OPEN ↑</td><td align="right"><b>92.67</b></td><td align="right">92.61</td><td align="right">89.55</td><td align="right">81.87</td></tr> <tr><td>AlpacaEval</td><td>OPEN ↑</td><td align="right">69.70</td><td align="right">52.87</td><td align="right"><b>71.14</b></td><td align="right">51.81</td></tr> <tr><td>AlpacaEval-Full</td><td>OPEN ↑</td><td align="right">70.75</td><td align="right">52.45</td><td align="right"><b>72.21</b></td><td align="right">52.43</td></tr> <tr><td>BBH</td><td>QA ↑</td><td align="right">44.18</td><td align="right">52.66</td><td align="right"><b>57.22</b></td><td align="right">43.80</td></tr> <tr><td>CommonEval</td><td>OPEN ↑</td><td align="right"><b>70.20</b></td><td align="right">55.41</td><td align="right">68.25</td><td align="right">55.49</td></tr> <tr><td>MMSU</td><td>QA ↑</td><td align="right"><b>43.67</b></td><td align="right">32.08</td><td align="right">43.48</td><td align="right">32.08</td></tr> <tr><td>OpenBookQA</td><td>QA ↑</td><td align="right">76.62</td><td align="right">76.12</td><td align="right"><b>91.54</b></td><td align="right">70.90</td></tr> <tr><td>SD-QA</td><td>QA ↑</td><td align="right">46.87</td><td align="right">25.89</td><td align="right"><b>48.23</b></td><td align="right">27.52</td></tr> <tr><td>WildVoice</td><td>OPEN ↑</td><td align="right"><b>66.88</b></td><td align="right">46.33</td><td align="right">66.25</td><td align="right">45.77</td></tr> <tr><td>Speech-IF</td><td>IFEval</td><td>OPEN ↑</td><td align="right">61.20</td><td align="right">44.88</td><td align="right"><b>67.61</b></td><td align="right">35.21</td></tr> <tr><td>Avg.</td><td>Speech-QA / IF</td><td>↑</td><td align="right">64.27</td><td align="right">53.13</td><td align="right"><b>67.55</b></td><td align="right">49.69</td></tr> <tr><td rowspan="3">Speech Generation</td><td rowspan="3">KVoiceBench (Subset-200)</td><td>CER ↓</td><td align="right"><b>1.86</b></td><td align="right">–</td><td align="right">26.37</td><td align="right">4.16</td></tr> <tr><td>UTMOS ↑</td><td align="right">2.871</td><td align="right">–</td><td align="right"><b>3.178</b></td><td align="right">3.171</td></tr> <tr><td>STOI ↑</td><td align="right">0.990</td><td align="right">–</td><td align="right"><b>0.999</b></td><td align="right">0.996</td></tr> </tbody> </table>

Averaged over the ten KVoiceBench subsets, A.X K2 ALM scores 64.27 — 95% of Qwen3-Omni-30B-A3B (67.55) at roughly two-thirds of its parameters — and outperforms Qwen2.5-Omni-7B and HyperCLOVA X 8B Omni by 21% and 29% (relative).

Demos

1. Natural Voice Conversation A fluid, natural spoken dialogue with the model. Real-time audio streaming on both input and output allows responses to begin almost immediately after the user finishes speaking. It also supports barge-in: if the user speaks while the model is still talking, it stops and starts listening right away.

<video controls playsinline preload="metadata" width="720"> <source src="https://cdn-uploads.huggingface.co/production/uploads/noauth/5CaJJY7leg7W8GWfDwyJs.mp4" type="video/mp4"> <source src="https://huggingface.co/skt/A.X-K2-ALM/resolve/main/assets/health_demo.mp4" type="video/mp4"> <a href="https://huggingface.co/skt/A.X-K2-ALM/resolve/main/assets/health_demo.mp4?download=true">Download the demo video</a> </video>

2. Context-Aware Turn-Taking The context-aware VAD ignores listener back-channels (e.g., "음", "아") and waits through mid-sentence hesitations — so it neither falsely stops nor cuts in, responding only once the user has actually finished.

<video controls playsinline preload="metadata" width="720"> <source src="https://cdn-uploads.huggingface.co/production/uploads/noauth/htPKhczYGj4Nnh4jYKddh.mp4" type="video/mp4"> <source src="https://huggingface.co/skt/A.X-K2-ALM/resolve/main/assets/vad_demo.mp4" type="video/mp4"> <a href="https://huggingface.co/skt/A.X-K2-ALM/resolve/main/assets/vad_demo.mp4?download=true">Download the demo video</a> </video>

Availability

Model weights are planned for public release (coming soon). For now, this repository serves as the model card and hosts the demo videos above and the technical report for reference.

License

Released under the h-research license.

Citation

If you use A.X K2 ALM in your research, please cite the technical report:

@techreport{axk2alm2026,
  title       = {A.X K2 ALM (Audio Language Model)},
  author      = {SKT A.X Team},
  year        = {2026},
  institution = {SK Telecom}
}

Contact

For A.X models for business applications or scalable deployment, please contact [email protected].