Voxtral-Mini-3B-2507-estonian

Voxtral Mini 3B 2507 Estonian

This model is a full-parameter fine-tune of mistralai/Voxtral-Mini-3B-2507 for Estonian speech recognition and audio understanding. It was trained on a mixture of transcription, subtitle and stenogram generation, news writing, speech translation, summarization, and question-answering tasks.

Model details

Intended uses

The model is intended for research and development involving Estonian speech, including:

• verbatim automatic speech recognition;
• readable subtitle and parliamentary stenogram generation;
• summaries and news stories based on broadcast audio;
• answering questions about spoken content; and
• translation of Estonian speech into English.

The output style depends on the instruction. Prompts should state the desired task and format explicitly. This is not a certified system for legal, medical, safety-critical, or high-stakes transcription.

Usage with Transformers

The following example uses the selected local checkpoint for Estonian transcription:

import torch
from transformers import AutoProcessor, VoxtralForConditionalGeneration

model_id = "exp/models/voxtral-mini-3b-2507-full/1c/checkpoint-4500"
device = "cuda"

processor = AutoProcessor.from_pretrained(model_id)
model = VoxtralForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="sdpa",
    low_cpu_mem_usage=True,
).to(device)
model.eval()

inputs = processor.apply_transcription_request(
    language="et",
    audio="example.wav",
    model_id=model_id,
    return_tensors="pt",
)
inputs = {
    key: value.to(device=device, dtype=torch.bfloat16)
    if value.is_floating_point()
    else value.to(device)
    for key, value in inputs.items()
}

with torch.inference_mode():
    output_ids = model.generate(**inputs, do_sample=False, max_new_tokens=8192)

prompt_length = inputs["input_ids"].shape[1]
text = processor.batch_decode(
    output_ids[:, prompt_length:], skip_special_tokens=True
)[0]
print(text.strip())

Use 16 kHz mono audio where possible. For long recordings, segmenting at natural speech boundaries is recommended.

Training data

The training data was combined from seven task datasets.

Of the 7,035 filtered examples, 5,500 exceeded the task-specific audio-duration limit and 1,535 exceeded the maximum combined audio/text sequence length. Transcription examples were limited to 30 minutes and audio-understanding examples to 40 minutes. The final prepared manifest used a maximum sequence length of 28,000 tokens.

Combined sequence lengths in the retained manifest were:

Some instructions and answers in trainining data and the English news-story targets were generated with google/gemini-3.1-flash-lite. Consequently, the training targets may contain errors or stylistic biases inherited from that model.

The source collections include ERR broadcasts and subtitles, an Estonian ASR manifest, ERR news video/audio and article text, Päevakaja programmes, Riigikogu recordings and stenograms, and TalTechNLP/qa_broadcast_conv_et. Users are responsible for checking the terms, copyright, privacy constraints, and permitted uses of the underlying data for their application.

Training procedure

Training used four NVIDIA A100-SXM4 80 GB GPUs. The per-device batch size was 1 with 8 gradient-accumulation steps, giving an effective global batch size of 32. Both the audio encoder and aligner were unfrozen.

The run was resumed from checkpoints during development, and the configured maximum sequence length was reduced from 32,768 to 30,000 and finally 28,000. The data statistics above describe the final prepared 28,000-token manifest; they should not be interpreted as an exact record of the manifest state at every earlier resumed stage.

Evaluation

Checkpoints were ranked by word error rate on an internal held-out Estonian ASR set. Generation used greedy decoding (temperature=0) with up to 8,192 new tokens.

The validation set contains only 10 recordings. These figures are useful for checkpoint selection but should not be treated as a broad estimate of Estonian ASR quality or as directly comparable with public benchmark results.

Toolkits and environment

The recorded environment for the final training stage included:

A local ms-swift plugin supplied Voxtral dataset integration and support for the repository's ark:<path>:<offset> audio references.

Limitations

• The model can hallucinate, omit, repeat, or incorrectly normalize speech.
• Performance may degrade for dialects, code-switching, overlapping speakers, noise, distant microphones, music, or domains underrepresented in training.
• The English outputs were trained on machine-generated references and may inherit translation errors.
• Generated summaries and answers are not guaranteed to be factually faithful to the recording.
• Training data may contain personal names, opinions, and other information present in public broadcasts or parliamentary speech. Outputs should be reviewed before publication or consequential use.

Acknowledgements

The model builds on Mistral AI's Voxtral Mini 3B 2507. See the upstream model card for architecture details, original capabilities, usage requirements, and base-model limitations.