Downloads · 30 days
84
20% of all-time downloads
leophill/w2v-bert-2.0-afrivoices-ea-6l-asr-int8-edge
w2v-bert-2.0-afrivoices-ea-6l-asr-int8-edge is a automatic speech recognition model from leophill. Use it when you need speech turned into text. It is set up for ctranslate2. The card lists the license as cc-by-4.0.
Edge conversions of leophill/w2v-bert-2.0-afrivoices-ea-6l-asr : CTranslate2 int8 (weight mapping — CTranslate2 implements the Wav2Vec2-BERT architecture natively) and ONNX int8 (graph export with the CTC head baked i…
Downloads · 30 days
84
20% of all-time downloads
All-time downloads
420
Public
Repo size
4.7 GB
Likes
2
Public
Click a slice to open those files.
.data2.4 GB · 59%
From the Hugging Face model README
Edge conversions of leophill/w2v-bert-2.0-afrivoices-ea-6l-asr : CTranslate2 int8 (weight mapping — CTranslate2 implements the Wav2Vec2-BERT architecture natively) and ONNX int8 (graph export with the CTC head baked in, dynamically quantised).
CTranslate2 is the recommended engine (fastest, smallest RSS); ONNX is provided for runtimes where CTranslate2 is unavailable (onnxruntime is broadly portable, incl. mobile).
One 580M-parameter model transcribes all six languages — Dholuo (luo), Kalenjin (kln), Kikuyu (kik), Maasai (mas), Somali (som), Swahili (swa) — over a shared 34-character union vocabulary (Kikuyu ĩ/ũ preserved). No language token or language input exists: the model transcribes whichever of the six languages it hears.
Built for the AfriVoices East Africa ASR Hackathon (Kaggle; Digital Umuganda & Maseno Center for Applied Artificial Intelligence) — 2nd place (provisional), macro-WER 0.34819 — and its edge constraints: <1B parameters, CPU inference within 8 GB RAM, near-real-time transcription.
Recent CTranslate2 releases (measured at 4.8.1) return encoder hidden
states (T, 1024) from encode() — not CTC logits. This repository ships
lm_head.npz (the 1024→34 CTC projection, fp32, ~140 KB); apply it after
encode() as shown below. Older CTranslate2 builds that emit (T, 34)
logits directly need no projection.
import re, numpy as np, soundfile as sf, ctranslate2
from transformers import SeamlessM4TFeatureExtractor, Wav2Vec2CTCTokenizer
repo = "leophill/w2v-bert-2.0-afrivoices-ea-6l-asr-int8-edge" # this repo
model = ctranslate2.models.Wav2Vec2Bert(
repo, device="cpu", compute_type="int8", intra_threads=4)
fe = SeamlessM4TFeatureExtractor.from_pretrained(repo)
tok = Wav2Vec2CTCTokenizer.from_pretrained(repo)
audio_file = "audio.wav"
audio, sr = sf.read(audio_file) # 16 kHz mono
feats = fe(audio, sampling_rate=sr, return_tensors="np").input_features
out = np.asarray(model.encode(
ctranslate2.StorageView.from_array(feats.astype(np.float32)),
to_cpu=True))[0]
if out.shape[-1] != 34: # CT2 >= 4.8: hidden states
z = np.load("lm_head.npz")
out = out @ z["weight"].T + z["bias"]
text = tok.decode(out.argmax(-1), group_tokens=True) # greedy CTC
text = re.sub(" +", " ", text).strip() # clean extra spaces
print(text)
import re, numpy as np, soundfile as sf, onnxruntime as ort
from transformers import SeamlessM4TFeatureExtractor, Wav2Vec2CTCTokenizer
from huggingface_hub import hf_hub_download
repo = "leophill/w2v-bert-2.0-afrivoices-ea-6l-asr-int8-edge" # this repo
onnx_path = hf_hub_download(repo, "onnx/model_int8.onnx") # cached locally
sess = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
fe = SeamlessM4TFeatureExtractor.from_pretrained(repo)
tok = Wav2Vec2CTCTokenizer.from_pretrained(repo)
audio_file = "audio.wav"
audio, sr = sf.read(audio_file) # 16 kHz mono
enc = fe(audio, sampling_rate=sr, return_tensors="np")
mask = enc.get("attention_mask",
np.ones(enc.input_features.shape[:2], dtype=np.int64))
logits = sess.run(["logits"], {
"input_features": enc.input_features.astype(np.float32),
"attention_mask": np.asarray(mask, dtype=np.int64),
})[0][0] # (T, 34) — head is in the graph
# (quantisation covers MatMul/Gemm only — no ConvInteger nodes)
text = tok.decode(logits.argmax(-1), group_tokens=True) # greedy CTC
text = re.sub(" +", " ", text).strip() # clean extra spaces
print(text)
For best accuracy with either engine, decode with pyctcdecode + a per-language 5-gram KenLM (beam width 100; the hackathon submission used per-language α ∈ {0.25, 0.5}, β = 0.0, Swahili greedy) and window long audio at 28 s with 4 s overlap. The complete decode pipeline (windowing, stitching, LM configs) is in the solution repository.
Conditions: this checkpoint, identical clips, identical pyctcdecode + KenLM decode path, x86 CPU, Somali 200-clip long-form-skewed sample, 4 threads:
| engine | ΔWER vs fp32 | RTF total | peak RSS |
|---|---|---|---|
| torch fp32 (reference) | — | 5.9 | 5.1 GB |
| CTranslate2 int8 | −0.003 | 0.412 | 2.7 GB |
| ONNX Runtime int8 | −0.003 | 0.449 | 4.7 GB |
int8 costs no measurable WER (Δ within beam tie-breaking noise, two independent engines agreeing).
Full-test-set validation (4 threads per process — one edge device's budget): per-language RTF 0.51–0.64 (worst: Kalenjin, largest-lexicon LM); worst-case peak RSS 6.2 GB (Somali, longest clips) using sequential shard recycling to bound allocator growth — within the 8 GB envelope. Measured on a Galaxy S21: RTF 1.308 (Somali sample).
16 kHz mono speech in the six covered languages; performance varies strongly by register (scripted read speech vs long-form spontaneous) and follows the training-data conventions of the hackathon corpora. Not a language identifier; audio outside the six languages produces unusable text.
@misc{w2v-bert-2.0-afrivoices-ea-6l-asr,
author = {Leopold Hillah},
title = {Jointly Finetuning Wav2Vec-BERT 2.0 for 6 East African Languages},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/leophill/w2v-bert-2.0-afrivoices-ea-6l-asr}
}
@misc{afri-voices-east-africa-asr-hackathon,
author = {{Digital Umuganda} and {Maseno University} and
{Maseno Center for Applied Artificial Intelligence}},
title = {AfriVoices East Africa: ASR Hackathon},
year = {2026},
howpublished = {\url{https://kaggle.com/competitions/afri-voices-east-africa-asr-hackathon}},
note = {Kaggle}
}