data2vec-audio-960h-GGUF

Data2Vec Audio (GGUF)

GGUF conversion of facebook/data2vec-audio-base-960h for use with CrispASR.

Model Details

• Architecture: Data2Vec Audio — wav2vec2-style CNN (7L, 512-dim) + 12-layer transformer (768-dim, 12 heads) + CTC head
• Parameters: ~95M
• Training: Self-supervised pre-training on LibriSpeech 960h, fine-tuned with CTC loss
• Language: English only
• License: Apache 2.0
• WER: 1.89% (LibriSpeech test-clean), 4.07% (test-other)

Usage with CrispASR

# Uses the wav2vec2 backend (auto-detected from GGUF architecture)
crispasr --backend wav2vec2 -m data2vec-audio-base-960h-q4_k.gguf -f audio.wav

Architecture Notes

Data2Vec Audio differs from standard wav2vec2 in three ways handled by the converter:

1. 5-layer positional convolution (vs 1 for wav2vec2), each with Conv1d + LayerNorm(no affine) + GELU
2. Global encoder LayerNorm BEFORE transformer layers (vs after for wav2vec2)
3. POST-norm encoder despite using LayerNorm in CNN (wav2vec2-large uses pre-norm)

All three are auto-detected from the HuggingFace model config and stored as GGUF metadata flags.

Files

Accuracy

Tested on JFK inaugural address (11s):

AND SO A MY FELLOW AMERICANS ASK NOT WHAT YOUR COUNTRY CAN DO FOR YOU
ASK WHAT YOU CAN DO FOR YOUR COUNTRY

Identical to the Python HuggingFace reference output. All quantized variants produce the same transcription.

Citation

@inproceedings{baevski2022data2vec,
  title={data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language},
  author={Baevski, Alexei and Hsu, Wei-Ning and Xu, Qiantong and Babu, Arun and Gu, Jiatao and Auli, Michael},
  booktitle={ICML},
  year={2022}
}

Provenance and EU AI Act Art. 53 note

• Upstream model: facebook/data2vec-audio-base-960h — published by facebook.
• Upstream licence: apache-2.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.