hanziflow-v2-1-multihead-wav2vec2

HanziFlow V2.1 Multi-Head Wav2Vec2

This is the private research checkpoint for the HanziFlow V2.1 Mandarin pronunciation model.

The model uses a shared 24-layer XLSR-53 Wav2Vec2 encoder and three independent CTC heads:

• Mandarin initial: 24 classes
• Mandarin final: 44 classes
• Mandarin tone: 7 classes

Each vocabulary includes CTC blank ID 0 and unknown ID 1.

Base encoder

• Model: jonatasgrosman/wav2vec2-large-xlsr-53-chinese-zh-cn
• Revision: 9c9f6c082674a183353d6193a5ca09f100ae1b26
• Hidden size: 1024
• Transformer layers: 24
• Attention heads: 16
• Sampling rate: 16 kHz
• Original character CTC head: discarded
• Convolutional feature extractor: frozen during training

Training data

• Corpus: AISHELL-1
• Training utterances: 14,285
• Development utterances: 1,549
• Training speakers: 310
• Development speakers: 36
• Train/dev speaker overlap: 0
• Total train/dev audio: 17.714 hours
• Total target syllables: 206,000

Only records physically present in the reconstructed AISHELL train and development partitions were retained. Another 817 records were quarantined.

Surface-pronunciation labels

The targets represent sentence-level Mandarin surface pronunciation. Conservative sandhi rules were applied only to approved contexts. Ambiguous cross-word sandhi, long third-tone sequences, uncertain 一 and 不 contexts, erhua, and ambiguous neutral-tone cases were held out.

The corrected V2.1 policy preserves na4 bian1 for 那边, na4 li3 for 那里, xiao3 peng2 you3 for 小朋友, zhe4 bian1 for 这边, and contextual zhi1/zhi3 labels for 只.

Development results

The selected checkpoint is optimizer step 2,679 after three epochs.

The selected combined development CTC loss was approximately 0.103215.

These are development-set component error rates. They must not be interpreted as real-world learner pronunciation accuracy.

Loading

from transformers import AutoModel
from transformers import Wav2Vec2FeatureExtractor

repo_id = "thin-nwe-soe/hanziflow-v2-1-multihead-wav2vec2"

feature_extractor = (
    Wav2Vec2FeatureExtractor.from_pretrained(repo_id)
)

model = AutoModel.from_pretrained(
    repo_id,
    trust_remote_code=True,
)

model.eval()

For a batch of 16 kHz mono waveforms:

inputs = feature_extractor(
    waveforms,
    sampling_rate=16_000,
    padding=True,
    return_tensors="pt",
)

outputs = model(
    input_values=inputs.input_values,
    attention_mask=inputs.attention_mask,
)

initial_logits = outputs.initial_logits
final_logits = outputs.final_logits
tone_logits = outputs.tone_logits

The three CTC heads decode independently, so their predicted sequence lengths may differ. The downstream pronunciation-scoring pipeline must align the components.

Intended use

• HanziFlow research and development
• Mandarin initial/final/tone recognition
• Pronunciation-evaluation pipeline development
• Controlled comparison with HanziFlow V1

The model is not yet validated as a standalone learner-pronunciation scoring system.

Evaluation limitations

AISHELL test audio was not used or scanned during the corrected V2 preparation and training pipeline. However, earlier upstream project experiments exposed AISHELL test transcripts. AISHELL test must therefore not be treated as a fully pristine external benchmark.

External non-native evaluation with LATIC and/or OMPAL is still required before deployment claims are made.

Privacy and release status

Keep this repository private until external evaluation, pipeline validation, attribution review, and release review are complete.

No AISHELL waveform files or sentence-level training manifests are included.

License and attribution

The base checkpoint and AISHELL-1 are published under the Apache License 2.0. Retain their attribution when redistributing the model.

This repository contains custom model code. Loading with Transformers requires trust_remote_code=True.