Downloads · 30 days
0
marcosremar2/minimind-o-ptbr-talker-curriculum-100k-poc
minimind-o-ptbr-talker-curriculum-100k-poc is a text-to-audio model from marcosremar2. Use it for the text-to-audio task on the model card, and read the license before you ship it in a product. The card lists the license as other.
Experimental repository for the PT-BR speech fine-tuning work around MiniMind-O, Tucano2 1.5B and Mimi codec tokens.
Downloads · 30 days
0
Access
Public
Updated May 23, 2026
Repo size
1.5 GB
Likes
0
Public
Click a slice to open those files.
.pt738 MB · 49%
From the Hugging Face model README
Experimental repository for the PT-BR speech fine-tuning work around MiniMind-O, Tucano2 1.5B and Mimi codec tokens.
This is not production-ready. The current result proves that the pipeline can connect text, Tucano2 hidden states, a Mimi-token Talker and Mimi decoding, but end-to-end speech for new Tucano2 responses is still unstable.
Build a small Portuguese voice assistant stack:
Whisper tiny/base
→ Tucano2 1.5B Q4 as Portuguese Thinker
→ small Mimi Talker
→ Mimi decoder
The near-term target is a Talker that can convert short PT-BR tutor/dialogue responses into intelligible Mimi audio tokens.
Original MiniMind-O native checkpoints from the earlier PT-BR text/T2A curriculum:
checkpoints/sft_ptbr_curriculum_100k_textfix_768.pth
checkpoints/sft_ptbr_curriculum_100k_t2a_repair_768.pth
checkpoints/sft_ptbr_curriculum_100k_mixed_768.pth
Dataset artifacts:
data/ptbr_curriculum_100k.jsonl
data/sft_text_ptbr_curriculum_100k.parquet
Latest experimental Talker checkpoints:
checkpoints_tucano2_mimi/tucano_hidden_diagonal_bridge13_8k_h768_l8_duration_stop_heads_only.pt
checkpoints_tucano2_mimi/tucano_hidden_diagonal_bridge13_8k_h768_l8_prev_noise_from_duration_stop.pt
The best current checkpoint for duration control is:
checkpoints_tucano2_mimi/tucano_hidden_diagonal_bridge13_8k_h768_l8_duration_stop_heads_only.pt
The prev_noise_from_duration_stop checkpoint is an experimental hardening run. It did not improve the end-to-end result yet, but it is included for reproducibility.
Tools:
tools_tucano2_mimi/talker_text_mimi/
Evaluation WAVs and manifests:
eval_tucano2_mimi/
The current Tucano2 → Mimi Talker uses:
duration_head to predict how many Mimi frames to generate.stop_head to predict where free-run generation should stop.prev-code noise training option to harden free-run generation against previous-token errors.Training only the new duration/stop heads while freezing the audio body worked as intended:
trainable parameters: 126,114 / 92,219,042
duration_mae: about 56 frames → about 3-5 frames
stop_mae: about 49 positions → about 2-4 positions
On offset-5000 diagnostic samples, predicted free-run lengths became close to target:
37 target frames → 40 generated frames
35 target frames → 34 generated frames
45 target frames → 46 generated frames
But the free-run audio still drifts in content.
Implemented and tested:
--prev-code-noise-prob 0.03
--prev-code-noise-mode random
The training run stayed numerically healthy, but the end-to-end Whisper transcripts were still not acceptable. This means the next bottleneck is not duration; it is autoregressive code stability/content preservation.
The full chain runs:
prompt
→ Tucano2 generated text
→ Tucano2 response hidden states
→ Mimi Talker
→ Mimi decoder
→ WAV
→ Whisper validation
Quality status:
technical pipeline: passes
duration control: improved
teacher-forced audio: often good
free-run audio: still unstable
new Tucano2 responses: not production usable
These commands assume the original workspace layout:
/workspace/minimind-o
Train the duration/stop heads only:
python tools_ptbr/talker_text_mimi/train_tucano_hidden_mimi_talker_diagonal_ss.py \
--from-checkpoint out_talker_text_mimi/tucano_hidden_diagonal_bridge13_8k_h768_l8_tf6000_ss3000_p005/talker_tucano_hidden.pt \
--cache-index data_ptbr/talker_text_mimi/tucano_hidden_cache_bridge13_16k/index.jsonl \
--train-jsonl data_ptbr/talker_text_mimi/tucano2_contextual_text_mimi_full.jsonl \
--out-dir out_talker_text_mimi/tucano_hidden_diagonal_bridge13_8k_h768_l8_duration_stop_heads_only \
--limit-examples 16384 \
--max-steps 2000 \
--batch-size 16 \
--hidden-size 768 \
--decoder-layers 8 \
--heads 12 \
--learning-rate 1e-3 \
--audio-loss-weight 0 \
--duration-loss-weight 0.20 \
--stop-loss-weight 0.05 \
--train-duration-stop-only \
--fp16
Train the prev-code-noise hardening run:
python tools_ptbr/talker_text_mimi/train_tucano_hidden_mimi_talker_diagonal_ss.py \
--from-checkpoint out_talker_text_mimi/tucano_hidden_diagonal_bridge13_8k_h768_l8_duration_stop_heads_only/talker_tucano_hidden.pt \
--cache-index data_ptbr/talker_text_mimi/tucano_hidden_cache_bridge13_16k/index.jsonl \
--train-jsonl data_ptbr/talker_text_mimi/tucano2_contextual_text_mimi_full.jsonl \
--out-dir out_talker_text_mimi/tucano_hidden_diagonal_bridge13_8k_h768_l8_prev_noise_from_duration_stop \
--limit-examples 8192 \
--max-steps 1200 \
--batch-size 4 \
--hidden-size 768 \
--decoder-layers 8 \
--heads 12 \
--learning-rate 1e-5 \
--ss-prob 0.02 \
--prev-code-noise-prob 0.03 \
--prev-code-noise-mode random \
--duration-loss-weight 0.02 \
--stop-loss-weight 0.01 \
--fp16
Run end-to-end evaluation:
python tools_ptbr/talker_text_mimi/e2e_tucano_to_mimi_audio.py \
--talker-checkpoint out_talker_text_mimi/tucano_hidden_diagonal_bridge13_8k_h768_l8_duration_stop_heads_only/talker_tucano_hidden.pt \
--out-dir eval_ptbr/e2e_tucano2_h768_mimi_duration_stop_heads_only \
--prompt "Como eu peço um café de forma educada?" \
--prompt "Me corrija se eu disser: eu vai ao mercado." \
--prompt "Me dê uma frase curta para praticar o som do R." \
--use-duration-head \
--use-stop-head \
--whisper-model base
The next real step is not more duration training. It is content-stability training for free-run Mimi code generation:
.pth files, not standalone Transformers model directories.See SHA256SUMS.txt and file_manifest.json for file sizes and checksums.