Downloads · 30 days
10
9% of all-time downloads
GaborMadarasz/gemma-4-E2B-it-CV24-VP-FL_hun
gemma-4-E2B-it-CV24-VP-FL_hun is a automatic speech recognition model from GaborMadarasz. Use it when you need speech turned into text. It is set up for transformers. The card lists the license as gemma.
Fine-tuned google/gemma-4-E2B-it for Hungarian automatic speech recognition (ASR) using LoRA adapters, trained with Unsloth.
Downloads · 30 days
10
9% of all-time downloads
All-time downloads
113
Public
Parameters
5.1B
10.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors10.2 GB · 100%
From the Hugging Face model README
Fine-tuned google/gemma-4-E2B-it for Hungarian automatic speech recognition (ASR) using LoRA adapters, trained with Unsloth.
This model is a LoRA fine-tune of Gemma 4 E2B (2B parameter Mixture-of-Experts multimodal variant), adapted for Hungarian speech transcription using the multimodal chat format. The audio encoder and language model backbone are fine-tuned jointly via low-rank adaptation.
google/gemma-4-E2B-itFastModelpush_to_hub_merged)On 448 held-out random sample
| Metric | Value |
|---|---|
| Test WER | 0.2009 (20.09%) |
| Test CER | 0.0583 (5.83%) |
| Parameter | Value |
|---|---|
r | 16 |
lora_alpha | 16 |
lora_dropout | 0 |
bias | none |
use_rslora | False |
| Target modules (language) | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Target modules (audio) | post, linear_start, linear_end, embedding_projection, ffw_layer_1, ffw_layer_2, output_proj |
finetune_vision_layers | False |
finetune_language_layers | True |
finetune_attention_modules | True |
finetune_mlp_modules | True |
| Parameter | Value |
|---|---|
per_device_train_batch_size | 2 |
gradient_accumulation_steps | 4 |
effective_batch_size | 8 |
num_train_epochs | 1 |
learning_rate | 8e-5 |
lr_scheduler_type | cosine |
warmup_ratio | 0.03 |
optimizer | adamw_8bit |
weight_decay | 0.001 |
max_seq_length | 1024 |
seed | 3407 |
load_in_4bit | True (training-time quantization) |
Each sample is formatted into a multimodal chat conversation: <bos><|turn>system You are an assistant that transcribes speech accurately.<turn|> <|turn>user <|audio|>Please transcribe this audio.<turn|> <|turn>model [transcription]<turn|>
Single RTX3060 12GB GPU
598.12 minutes used for training.
Peak reserved memory = 9.73 GB.
from unsloth import FastModel
from transformers import TextStreamer
model, processor = FastModel.from_pretrained(
model_name="GaborMadarasz/gemma-4-E2B-it-CV24_hun",
max_seq_length=1024,
load_in_4bit=True,
)
def transcribe(audio_array):
messages = [
{
"role": "system",
"content": [{"type": "text", "text": "You are an assistant that transcribes speech accurately."}],
},
{
"role": "user",
"content": [
{"type": "audio", "audio": audio_array},
{"type": "text", "text": "Please transcribe this audio."}
]
}
]
model.generate(
**processor.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to("cuda"),
max_new_tokens=256,
do_sample=False,
streamer=TextStreamer(processor, skip_prompt=True),
)
Note: Audio input must be resampled to 16,000 Hz before passing to the processor.
load_in_4bit=True; the merged float16 weights are provided for full-precision inference.UnslothVisionDataCollatortransformers==5.5.0