Downloads · 30 days
9
12% of all-time downloads
hhim8826/qwen3-asr-lora-ja-anime
qwen3-asr-lora-ja-anime is a automatic speech recognition model from hhim8826. Use it when you need speech turned into text. It is set up for peft.
Qwen/Qwen3-ASR-1.7B-hf 的 LoRA 微調版本, 針對日文動漫配音的說話風格、腔調與用詞習慣做適應,訓練資料來自 hhim8826/japanese-anime-speech-v2-split。
Downloads · 30 days
9
12% of all-time downloads
All-time downloads
76
Public
Repo size
243 MB
Likes
0
Public
Click a slice to open those files.
.pt51.6 MB · 45%
From the Hugging Face model README
Qwen/Qwen3-ASR-1.7B-hf 的 LoRA 微調版本, 針對日文動漫配音的說話風格、腔調與用詞習慣做適應,訓練資料來自 hhim8826/japanese-anime-speech-v2-split。
只微調語言模型 decoder 的 attention 投影層,音訊 encoder(audio_tower)維持凍結不動。
target_modules 只涵蓋 language_model 的
q_proj / k_proj / v_proj / o_proj,不含 audio_towerimport torch
from peft import PeftModel
from transformers import AutoProcessor, AutoModelForMultimodalLM
base_model_id = "Qwen/Qwen3-ASR-1.7B-hf"
adapter_id = "hhim8826/qwen3-asr-lora-ja-anime"
processor = AutoProcessor.from_pretrained(adapter_id)
model = AutoModelForMultimodalLM.from_pretrained(base_model_id, dtype=torch.bfloat16, device_map={"": 0})
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()
inputs = processor.apply_transcription_request(audio="path/to/audio.wav", language="Japanese")
inputs = {k: v.to(model.device) for k, v in inputs.items()}
with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
generated = model.generate(**inputs, max_new_tokens=200, repetition_penalty=1.3, no_repeat_ngram_size=3)
text = processor.tokenizer.decode(
generated[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True
)
print(text.split("<asr_text>", 1)[-1].strip())
repetition_penalty 與 no_repeat_ngram_size 使用(見上方範例)。訓練資料集 japanese-anime-speech-v2-split 的授權為 GPL,使用或散布本 adapter 前請自行確認授權相容性。