whisper-large-v3-hungarian-lora

Whisper Large-v3 — Hungarian LoRA (CTranslate2)

  If this Hungarian ASR model helps your project, a coffee keeps the GPUs training. 🙏

A LoRA fine-tuned Whisper large-v3 model optimized for the Hungarian language. This model reduces common recognition inaccuracies and hallucinations of the base model on Hungarian speech (special Hungarian characters, native names, silence hallucinations).

This repository contains the CTranslate2-converted FP16 version for use with the popular faster-whisper library, plus the original PEFT LoRA adapter (in adapter/) for anyone who wants to merge onto the base model themselves — e.g. for MLX, vanilla transformers, or further fine-tuning.

Note: this model was previously published as whisper-large-v3-turbo-hungarian-lora and was renamed to reflect its true base — openai/whisper-large-v3 (32 decoder layers, 128 mel bins), not the 4-layer turbo variant. The old name still redirects here, so existing links and downloads keep working.

---

Highlights

• Optimized for Hungarian: LoRA fine-tuned (r=32, alpha=64, q_proj/v_proj) on Hungarian speech
• Fast inference: CTranslate2 engine; use compute_type="float16" (or "int8_float16" for lower VRAM)
• Two formats in one repo: ready-to-run CT2 weights + the raw PEFT adapter for custom workflows
• Perfect match for F5-TTS: recommended for generating accurate reference text (ref_text) for Maxdorger29/f5-tts-hungarian voice cloning

Quick Start (faster-whisper)

pip install faster-whisper

from faster_whisper import WhisperModel

model_id = "Maxdorger29/whisper-large-v3-hungarian-lora"
model = WhisperModel(model_id, device="cuda", compute_type="float16")

segments, info = model.transcribe("magyar_hang_minta.wav", language="hu", beam_size=5)

for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")

Lower VRAM: compute_type="int8_float16".

Using the LoRA adapter (transformers / PEFT)

The adapter/ folder contains the original PEFT artifacts (adapter_model.safetensors, ~60 MB). Merge onto the base model:

from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel

base = WhisperForConditionalGeneration.from_pretrained("openai/whisper-large-v3")
model = PeftModel.from_pretrained(base, "Maxdorger29/whisper-large-v3-hungarian-lora", subfolder="adapter")
merged = model.merge_and_unload()

processor = WhisperProcessor.from_pretrained("openai/whisper-large-v3")
merged.save_pretrained("./whisper-large-v3-hu-merged")
processor.save_pretrained("./whisper-large-v3-hu-merged")

MLX conversion (Apple Silicon)

After merging (above), convert the merged HF checkpoint with mlx-examples/whisper convert.py:

python convert.py --torch-name-or-path ./whisper-large-v3-hu-merged --mlx-path ./whisper-hu-mlx --dtype float16

Training Details

Repository Contents

Links

• Hugging Face (Model Weights): huggingface.co/Maxdorger29/whisper-large-v3-hungarian-lora
• GitHub (Issues & Discussion): github.com/Maxdorger/whisper-hungarian-lora

Companion Model (F5-TTS)

We originally fine-tuned this model as the transcription backend for Maxdorger29/f5-tts-hungarian, providing accurate reference texts (ref_text) for voice cloning.

Support the Project

Training and fine-tuning open-source TTS/STT models requires significant GPU resources. If this Hungarian-optimized Whisper LoRA speeds up your workflow, consider buying me a coffee — it directly supports the compute costs of future epochs! ☕

License

MIT License for the fine-tuning weights and scripts, provided usage does not violate the original terms of the OpenAI Whisper large-v3 model.

---

Fine-tuned for the Hungarian Open-Source AI Community.