room-voice-ai

RooM Voice AI — Model Weights Hub 🎙️

RooM Voice AI Model Weights Hub provides pre-trained, production-ready AI model checkpoints quantized in ONNX INT8 format, running purely on multi-threaded CPU AVX2 in real-time (0 MB GPU VRAM) for the RooM Voice AI (Voice Call Meeting AI) enterprise assistant system.

---

📦 Released Model Checkpoints

---

🧠 1. Technical Specification: PhoBERT-Joint Multi-Task NLU (Module 4)

The phobert_joint_meeting_nlu.onnx model is built on top of vinai/phobert-base-v2 with a Dual-Head architecture, simultaneously resolving both Vietnamese meeting dialogue semantic understanding tasks in a single forward pass:

1. Dialogue Intent Classification: 6 core enterprise meeting intent classes via a Softmax classification head.
2. Slot Entity Tagging: 11 BIO sequence tagging labels extracting personnel, timestamps, tasks, topics, and tools via a linear sequence classification head.

Multi-Task Joint Optimization Loss:

$$\mathcal{L}{\mathrm{joint}} = 0.5 \cdot \mathcal{L}{\mathrm{intent}}(\hat{y}{\mathrm{intent}}, y{\mathrm{intent}}) + 0.5 \cdot \mathcal{L}{\mathrm{slot}}(\hat{y}{\mathrm{slot}}, y_{\mathrm{slot}})$$

Core Meeting Intent Taxonomy:

• Summarize_Meeting / Create_Meeting_Summary — Meeting minutes summarization.
• Query_Project_Knowledge — Project architecture & documentation inquiry.
• Assign_Action_Item — Action item assignment & task delegation.
• Schedule_Followup / Schedule_Followup_Meeting — Follow-up meeting scheduling.
• Explain_Technical_Concept — Technical terminology & concept explanation.
• Chitchat_Clarification — General greeting & clarification.

Entity Slot Label Taxonomy (BIO Scheme):

• B-PER, I-PER (Assignee / Personnel name)
• B-TIME, I-TIME (Timestamp / Deadline deadline)
• B-TASK, I-TASK (Action item description / Task name)
• B-TOPIC, I-TOPIC (Meeting agenda / Discussion topic)
• B-TOOL, I-TOOL (Technical tools / Infrastructure: Jira, gRPC, Git, Docker, etc.)
• O (Outside / Non-entity filler words)

---

⚡ 2. Technical Specification: Wake-Word OpenWakeWord (Module 2)

The openwakeword_room.onnx model is specifically engineered for real-time Vietnamese meeting wake-word detection:

• Target Call Names (Wake-words): "AI Star Light", "Star Light ơi", "AI Trợ lý", "Hey Star Light", "Starlight ơi".
• Acoustic Input: 80-channel Log-Mel Spectrogram extracted from enhanced 16 kHz Mono Float32 PCM audio ($32\text{ ms} = 512\text{ samples}$ frame step).
• Circular Sliding RingBuffer: $1.5\text{ seconds}$ ($24,000\text{ samples}$ @ $16\text{ kHz}$).
• Temporal Posterior Smoothing: Moving Average Filter over $K = 4$ consecutive frames ($P_{\text{smooth}}(t) = \frac{1}{4}\sum_{i=0}^{3} P_{\text{raw}}(t-i)$).
• Dual-Threshold Hysteresis: Activation trigger threshold $\theta_{\text{wake}} = \theta_{\text{start}} = 0.75$, state reset threshold $\theta_{\text{reset}} = 0.40$.
• Refractory Debounce Lockout: $T_{\text{cooldown}} = 2.0\text{ seconds}$ ($N_{\text{debounce}} \approx 62\text{ frames}$ @ 32 ms) post-trigger to eliminate duplicate activations.
• Resource Footprint: CPU inference latency $\le 3.5\text{ ms/frame}$ on Intel Core i7, RAM $< 30\text{ MB}$, $0\text{ MB}$ GPU VRAM.

---

💻 3. Quickstart Guide (Python Code Snippets)

Dependency Installation:

pip install onnxruntime numpy transformers pyvi huggingface_hub

Inference with PhoBERT-Joint Multi-Task NLU:

import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer
from pyvi import ViTokenizer
from huggingface_hub import hf_hub_download

# 1. Download model and tokenizer from Hugging Face Hub
model_path = hf_hub_download(repo_id="sh1kaku/room-voice-ai", filename="phobert_joint_meeting_nlu.onnx")
tokenizer = AutoTokenizer.from_pretrained("sh1kaku/room-voice-ai", subfolder="tokenizer", use_fast=False)

# 2. Initialize ONNX Runtime Session on CPU AVX2
session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])

# 3. Preprocess input text with PyVi Vietnamese word segmentation
text = "Giao cho anh Tuấn kiểm tra lỗi kết nối API gRPC trước 17h chiều nay"
segmented_text = ViTokenizer.tokenize(text)
inputs = tokenizer(segmented_text, return_tensors="np", padding="max_length", max_length=128, truncation=True)

# 4. Execute Multi-Task Inference
ort_inputs = {
    "input_ids": inputs["input_ids"].astype(np.int64),
    "attention_mask": inputs["attention_mask"].astype(np.int64),
}
intent_logits, slot_logits = session.run(None, ort_inputs)

# 5. Decode Intent and BIO Slot Tags
INTENTS = [
    "Summarize_Meeting",
    "Query_Project_Knowledge",
    "Assign_Action_Item",
    "Schedule_Followup",
    "Explain_Technical_Concept",
    "Chitchat_Clarification",
]
SLOTS = [
    "O", "B-PER", "I-PER", "B-TIME", "I-TIME",
    "B-TASK", "I-TASK", "B-TOPIC", "I-TOPIC", "B-TOOL", "I-TOOL"
]

predicted_intent = INTENTS[np.argmax(intent_logits[0])]
predicted_slots = [SLOTS[idx] for idx in np.argmax(slot_logits[0], axis=-1)]

print(f"Input Text : {text}")
print(f"Intent     : {predicted_intent}")

Inference with Wake-Word Verification Model:

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download

# 1. Download Wake-Word model
ww_path = hf_hub_download(repo_id="sh1kaku/room-voice-ai", filename="openwakeword_room.onnx")
ww_session = ort.InferenceSession(ww_path, providers=["CPUExecutionProvider"])

# 2. Simulated Log-Mel feature tensor input (16 frames, 32 channels)
dummy_features = np.zeros((1, 16, 32), dtype=np.float32)
input_name = ww_session.get_inputs()[0].name
output_prob = ww_session.run(None, {input_name: dummy_features})[0]

print(f"Wake-Word Posterior Probability: {float(output_prob[0][0]):.4f}")

---

📊 4. Benchmark Evaluation & Performance Metrics

---

📜 5. Scientific Publication & Academic Citation

The underlying research and architecture behind these models are officially published on Zenodo:

• Paper Title: RooM: Real-Time Event-Driven Streaming Architecture for Vietnamese Voice Call Meeting AI
• Author: Vu, Ngoc Minh (Researcher) (Independent AI Researcher, Ho Chi Minh City, Vietnam)
• DOI: 10.5281/zenodo.22761612
• Zenodo Preprint: https://zenodo.org/records/22761612
• Academic License: Creative Commons Attribution 4.0 International (CC BY 4.0)

BibTeX Citation:

@article{vu2026room,
  title={RooM: Real-Time Event-Driven Streaming Architecture for Vietnamese Voice Call Meeting AI},
  author={Vu, Ngoc Minh},
  journal={Zenodo},
  year={2026},
  month={September},
  doi={10.5281/zenodo.22761612},
  url={https://doi.org/10.5281/zenodo.22761612}
}

---

🔗 Project & Repository Links

• GitHub Repository: https://github.com/sh1kaku59/room-voice-ai
• Zenodo Scientific Publication: https://zenodo.org/records/22761612
• Detailed Module Specifications: Doc/module2_specification.md & Doc/module4_specification.md
• License & Copyright: MIT License — Copyright © 2026 Vu, Ngoc Minh (sh1kaku59).