Downloads · 30 days
0
video-foley
jnwnlee/video-foley
video-foley is a machine learning model from jnwnlee. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-sa-4.0.
Official model checkpoint of "Video-Foley: Two-Stage Video-To-Sound Generation via Temporal Event Condition For Foley Sound".
Access
Public
Updated Dec 28, 2024
Repo size
4.4 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.pth4.1 GB · 93%
At a glance
- License
- cc-by-nc-sa-4.0
- Access
- Public
- Created
- Sep 28, 2024
- Updated
- Dec 28, 2024
- SHA
- eb8dfc5e
Model card
From the Hugging Face model README
Video-Foley
Official model checkpoint of "Video-Foley: Two-Stage Video-To-Sound Generation via Temporal Event Condition For Foley Sound".
(refer to the paper for further details)
Video2RMS
- files: checkpoint_000500_Video2RMS.pt (weight), opts.yml (config)
- training data: Greatest Hits (audio-visual, ~6hr)
RMS2Sound (including RMS-ControlNet)
- file: ControlNetstep300000.pth (weight)
- training data: FreeSound (audio only, ~6khr, in Wavcaps)
Citation
@article{video-foley,
title={Video-Foley: Two-Stage Video-To-Sound Generation via Temporal Event Condition For Foley Sound},
author={Lee, Junwon and Im, Jaekwon and Kim, Dabin and Nam, Juhan},
journal={arXiv preprint arXiv:2408.11915},
year={2024}
}
- License
- cc-by-nc-sa-4.0
- Created
- Sep 28, 2024
- Updated
- Dec 28, 2024