Downloads · 30 days
27
10% of all-time downloads
raniczan/wav2vec2-xls-r-300m-korean-english
wav2vec2-xls-r-300m-korean-english is a automatic speech recognition model from raniczan. Use it when you need speech turned into text. It is set up for transformers. The card lists the license as apache-2.0.
A jamo + Latin CTC acoustic model fine-tuned from facebook/wav2vec2-xls-r-300m for Korean / English code-switched speech. It emits per-frame character probabilities over a compact jamo + Latin vocabulary.
Downloads · 30 days
27
10% of all-time downloads
All-time downloads
267
Public
Parameters
316M
1.3 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors1.3 GB · 100%
From the Hugging Face model README
A jamo + Latin CTC acoustic model fine-tuned from facebook/wav2vec2-xls-r-300m
for Korean / English code-switched speech. It emits per-frame character
probabilities over a compact jamo + Latin vocabulary.
facebook/wav2vec2-xls-r-300m (300M multilingual wav2vec2), feature encoder frozen.The vocab is jamo (decomposed Hangul) + Latin, not raw syllables. You MUST decompose target text the same way, and recompose model output:
import unicodedata
def to_jamo(t): # text -> model target space (NFD on Hangul only)
return "".join(unicodedata.normalize("NFD", c) if "가" <= c <= "힣" else c for c in t)
def from_jamo(t): # model output -> readable text
return unicodedata.normalize("NFC", t)
import soundfile as sf, unicodedata
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
proc = Wav2Vec2Processor.from_pretrained("raniczan/wav2vec2-xls-r-300m-korean-english")
model = Wav2Vec2ForCTC.from_pretrained("raniczan/wav2vec2-xls-r-300m-korean-english").eval()
wav, sr = sf.read("clip.wav") # resample to 16 kHz first if needed
x = proc(wav, sampling_rate=16000, return_tensors="pt").input_values
ids = model(x).logits.argmax(-1)[0]
print(unicodedata.normalize("NFC", proc.decode(ids)))