Downloads · 30 days
12
31% of all-time downloads
jatshi/LLM-Guided-Speech-Enhancement-v4.1-GRPO
LLM-Guided-Speech-Enhancement-v4.1-GRPO is a audio-text-to-text model from jatshi. Use it for the audio-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.
This repository contains the promotion-gate-selected GRPO LoRA adapter and audio projector from LLM-Guided Speech Enhancement 4.1. A frozen Whisper-small encoder is compressed into 16 continuous prefix tokens for Qwen…
Downloads · 30 days
12
31% of all-time downloads
All-time downloads
39
Public
Repo size
92.5 MB
Likes
0
Public
Click a slice to open those files.
.pt44.1 MB · 45%
From the Hugging Face model README
This repository contains the promotion-gate-selected GRPO LoRA adapter and audio projector from LLM-Guided Speech Enhancement 4.1. A frozen Whisper-small encoder is compressed into 16 continuous prefix tokens for Qwen2.5-1.5B-Instruct. The language model emits a strict JSON DSP prescription; it does not directly synthesize waveform samples.
Source, exact configuration, tests, and the Chinese failure analysis: Jatshi/llm-guided-speech-enhancement.
The v4.0 GRPO run collapsed because every candidate in every group received zero reward. Version 4.1 starts from the verified SFT adapter, keeps an SFT anchor, rejects saturated pre-training canaries, and continuously calibrates otherwise-valid action parameters against the deterministic synthetic-degradation target. The target is data-derived; no LLM judge is used.
| Promotion evidence | Result |
|---|---|
| Pre-training canary | 48/48 valid JSON; 12/12 non-saturated groups |
| Training | 300 optimizer steps; 1,200 groups |
| Sampled valid JSON | 99.1458% |
| Non-saturated groups | 99.8333% |
| Held-out predictions | 1,416/1,416 successful |
| Placeholder rate | 0% |
| SFT mean verifiable reward | 0.953821 |
| GRPO mean verifiable reward | 0.976320 |
| Promotion gate | passed |
All 1,416 held-out predictions were executed by the bounded DSP engine. Candidates that failed the SI-SDR safety condition were rolled back to the original noisy waveform.
| Metric | Result |
|---|---|
| Safety accept / rollback | 68.15% / 31.85% |
| Mean SI-SDR gain | +0.35468 dB |
| Mean PESQ gain | +0.04056 |
| Mean STOI gain | +0.00260 |
| Mean / P95 prescription latency | 786.15 / 873.03 ms |
All four SNR buckets (<5, 5–10, 10–20, and >=20 dB) have positive mean gains for all three
metrics. These are objective, reference-based benchmark measurements rather than listening-test
claims.
adapter/: PEFT LoRA adapter and tokenizer files;audio_projector.pt: Whisper-hidden-state to 16-token prefix projection;reports/: canary, training, prediction, generalization, acceptance, and environment evidence;README.md: this model card.The base Qwen and Whisper weights are not redistributed.
Use configs/native_audio_autodl_grpo_recovery_12gb.json from the source repository. The verified
runtime used Python 3.10.8, PyTorch 2.5.1, Transformers 4.48.3, PEFT 0.14.0, Accelerate 1.2.1, and
DeepSpeed 0.16.3 on one RTX 3080 Ti 12GB. DeepSpeed ZeRO-2 at world_size=1 is runtime integration
evidence, not a multi-GPU scaling claim.
The source code is MIT licensed. Use of the adapter and reconstructed data remains subject to the Qwen, Whisper, LibriSpeech, and ESC-50 terms.