Downloads · 30 days
12
3% of all-time downloads
USTM
naver-ai/USTM
USTM is a audio-to-audio model from naver-ai. Use it for the audio-to-audio task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Unified Speech-Text Model (USTM), a speech-text cross-modal pretrained model obtained by further training Mistral-7B-v0.1 using a proposed unified speech-text pretraining methodology.
All-time downloads
366
Public
Parameters
7.3B
29.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors29.3 GB · 100%
At a glance
- Task
- Audio-to-Audio
- Library
- transformers
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Nov 28, 2024
- Updated
- Nov 28, 2024
- SHA
- 37e886ef
Base models
Model card
From the Hugging Face model README
Model Card
<!-- Provide a quick summary of what the model is/does. -->Unified Speech-Text Model (USTM), a speech-text cross-modal pretrained model obtained by further training Mistral-7B-v0.1 using a proposed unified speech-text pretraining methodology.
Paralinguistics-Aware Speech-Empowered LLMs for Natural Conversation [NeurIPS 2024]
- Repository: https://github.com/naver-ai/usdm
- Paper: https://openreview.net/forum?id=NjewXJUDYq
- Project Page: https://unifiedsdm.github.io/
BibTeX
@inproceedings{
kim2024paralinguisticsaware,
title={Paralinguistics-Aware Speech-Empowered Large Language Models for Natural Conversation},
author={Heeseung Kim and Soonshin Seo and Kyeongseok Jeong and Ohsung Kwon and Soyoon Kim and Jungwhan Kim and Jaehong Lee and Eunwoo Song and Myungwoo Oh and Jung-Woo Ha and Sungroh Yoon and Kang Min Yoo},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=NjewXJUDYq}
}
- Task
- Audio-to-Audio
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Languages
- en
- Created
- Nov 28, 2024
- Updated
- Nov 28, 2024