MOSS-Transcribe-Diarize-0.9B-MLX-INT8

MOSS-Transcribe-Diarize 0.9B MLX INT8

Native MLX weights for offline, speaker-attributed transcription on Apple Silicon. The Whisper encoder and VQ adaptor remain FP16; the tied Qwen3 decoder uses 8-bit affine group-64 weights.

The model is offline, not streaming. Audio is encoded in 30-second chunks, then all audio embeddings are concatenated into one globally contextualized prompt. Transcript tokens are generated autoregressively after the complete recording is available.

speech-swift

speech transcribe meeting.wav \
  --engine moss \
  --backend mlx \
  --model aufklarer/MOSS-Transcribe-Diarize-0.9B-MLX-INT8 \
  --kv-cache fp16

--kv-cache int8 reduces long-context memory separately from decoder weight quantization. Use FP16 cache when comparing INT5 and INT8 model quality. INT4 KV cache is not exposed because it failed the structured-output quality gate.

Matched INT5 / INT8 validation

Release build on an Apple M5 Pro with a FP16 KV cache. ASR uses 80 English-US FLEURS clips (759.56 seconds); diarization uses five VoxConverse test recordings (2,346.56 seconds), a 0.25-second collar, and 10 ms resolution. The diarization files are from ggfox00000/dia-voxconverse-test, not the Community-1 development slice used by the separate CoreML benchmark.

INT5 is the smaller default; INT8 is the slight ASR-quality reference. The five-file DER result supports parity, not a broad diarization-quality claim. Exact metrics and fixture IDs are recorded in validation.json.

Export provenance

The bundle is produced by scripts/export_moss_mlx.py in speech-swift from the immutable source revision above. export_config.json records tensor counts, checksums, quantization verification, and the runtime contract.

No training or fine-tuning was performed.

Limitations

The upstream project reports support for 50+ languages and recordings up to about 90 minutes, but does not publish an exhaustive language list. This bundle's matched validation covers English only. Actual duration is constrained by unified memory, KV-cache precision, transcript length, and the 131,072-token combined context. Speaker IDs are anonymous per recording. Timestamps may overlap or be malformed, and quantization can affect both word accuracy and speaker attribution.

License

Apache License 2.0, inherited from the upstream model.