Downloads · 30 days
46
20% of all-time downloads
llama_asr_tts_35000
Vikhrmodels/llama_asr_tts_35000
llama_asr_tts_35000 is a text generation model from Vikhrmodels. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
SALT is a multimodal model that extends pre-trained large language models (LLMs) by incorporating new audio tokens to handle both Text-to-Speech (TTS) and Automatic Speech Recognition (ASR) tasks. Our approach bridges…
All-time downloads
231
Public
Parameters
1.1B
4.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Sep 2, 2024
- Updated
- Sep 5, 2024
- SHA
- 067b45a7
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
SALT - Speech and Language Transformer
SALT is a multimodal model that extends pre-trained large language models (LLMs) by incorporating new audio tokens to handle both Text-to-Speech (TTS) and Automatic Speech Recognition (ASR) tasks. Our approach bridges the gap between text and audio modalities by expanding the model’s vocabulary with audio representations using SpeechTokenizer.
Unlike existing models that require full retraining or rely on adapters, SALT leverages pre-trained LLM knowledge while fine-tuning for speech-specific tasks.
After resolving training challenges through precision adjustments (e.g., tf32), SALT exhibits stable and effective performance across both TTS and ASR domains.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Sep 2, 2024
- Updated
- Sep 5, 2024