hindi-data2vec-aqc-gemma3-4b-it-lora

Hindi ASR — Data2Vec-AQC + Gemma-3-4B-IT (LoRA)

A Hindi Automatic Speech Recognition (ASR) model built using the SLAM-LLM framework.
It combines a frozen Data2Vec-AQC speech encoder with a Gemma-3-4B-IT language model fine-tuned via LoRA.

---

Model Architecture

LoRA Configuration

---

Training Details

---

WER Results

Evaluated on 6 standard Hindi ASR test sets using greedy decoding.

---

Usage

This repository contains only the LoRA adapter weights (pytorch_model.bin).
To run inference, you need:

1. Base LLM: google/gemma-3-4b-it
2. Speech Encoder: SPRING_INX_data2vec_aqc_Hindi.pt
3. SLAM-LLM framework

# Clone SLAM-LLM
git clone https://github.com/X-LANCE/SLAM-LLM

# Run inference
python examples/asr_librispeech/inference_asr_batch.py \
    ++model_config.llm_name=gemma-3-4b-it \
    ++model_config.llm_dim=2560 \
    ++model_config.encoder_name=data2vec_aqc \
    ++model_config.encoder_dim=1024 \
    ++model_config.encoder_projector=linear \
    ++model_config.encoder_projector_ds_rate=5 \
    ++dataset_config.prompt_style=gemma2 \
    ++ckpt_path=/path/to/pytorch_model.bin

---

Citation

If you use this model, please cite the SLAM-LLM framework:

@article{ma2024embarrassingly,
  title={An Embarrassingly Simple Approach for LLM with Strong ASR Capacity},
  author={Ma, Ziyang and others},
  journal={arXiv preprint},
  year={2024}
}