JoyAI-Echo

<p align="center"> <img src="assets/image.png" alt="Echo-LongVideo generated video gallery" width="100%"> </p> <div align="center"> <h1>Echo-LongVideo</h1> <p><strong>🎬 Pushing the Frontier of Long Video Generation</strong></p> <p>Official model weights for <strong>minute-level multi-shot audio-video generation</strong> with a distilled DMD generator, paired cross-modal memory, and story-level consistency.</p> <p> <a href="LongVideo_Tech_Report.pdf"><b>📄 Paper</b></a> | <a href="https://github.com/Echo-Team-Joy-Future-Academy-JD/JoyAI-Echo"><b>💻 Inference Code</b></a> | <a href="#model-details"><b>🧬 Model</b></a> | <a href="#usage"><b>🚀 Usage</b></a> | <a href="#results"><b>📊 Results</b></a> | <a href="#citation"><b>📝 Citation</b></a> </p> <p> <img src="https://img.shields.io/badge/Task-Text--to--Video-blue?style=flat-square" alt="Text-to-Video"> <img src="https://img.shields.io/badge/Modality-Audio%2BVideo-purple?style=flat-square" alt="Audio + Video"> <img src="https://img.shields.io/badge/Long%20Video-5%20min-d61f2c?style=flat-square" alt="5 minute long video"> <img src="https://img.shields.io/badge/Release-Model%20Weights-black?style=flat-square" alt="Model Weights"> </p> </div>

Model Summary

Echo-LongVideo (a.k.a. JoyAI-Echo) is a long-form, multi-shot, audio-video generation model. A cross-modal audio-visual memory bank preserves character appearance and voice timbre consistently across up to five-minute videos, and a post-training pipeline combining memory-based reinforcement learning with distribution matching distillation (DMD) delivers a 7.5× inference speedup without sacrificing quality.

In human evaluation, Echo-LongVideo decisively outperforms HappyOyster (directing mode) on long-form generation and surpasses the short-video specialist Wan 2.6 on human-centric tasks.

This repository hosts the released checkpoint. Inference code is released separately — see the Usage section.

Model Details

• Developed by: Echo Team @ Joy Future Academy, JD
• Model type: Text-to-(Audio+Video) diffusion transformer, DMD 8-step
• Modality: Text → synchronized video + audio
• Backbone: Built on top of LTX-Video
• Text encoder: google/gemma-3-12b-it (downloaded separately)
• Resolution / length (by default): 1280 × 736, 241 frames @ 25 fps per shot
• Max story length: up to 5 minutes (multi-shot)
• License: LTX-2 Community License Agreement

Highlights

• 🎞️ Minute-level multi-shot stories from a single prompt JSON.
• ⚡ DMD-distilled few-step inference, ~7.5× faster than the original pipeline.
• 🔊 Joint audio-video generation in a single pipeline.
• 🧠 Paired cross-modal memory bank for story-level identity and voice consistency.

Usage

Inference is run with the standalone Echo-LongVideo inference repository.

1. Download the checkpoint

huggingface-cli download <org>/Echo-LongVideo \
  --local-dir checkpoints

Also download the Gemma text encoder:

huggingface-cli download google/gemma-3-12b-it \
  --local-dir checkpoints/gemma-3-12b

Expected layout:

checkpoints/
├── echo-longvideo-release.safetensors
└── gemma-3-12b/

2. Get the inference code

git clone https://github.com/Echo-Team-Joy-Future-Academy-JD/JoyAI-Echo.git
cd JoyAI-Echo

Environment: Python 3.11 + PyTorch 2.8 + CUDA 12.8 (see the inference repo's environment.yml / requirements.txt).

3. Write a story prompt

Enhance your prompt first. We provide prompt enhancers — system prompts that expand a short story or idea into well-formed shot prompts: prompts/long_story_writer_system_prompt.md for long, multi-shot video, and prompts/short_story_writer_system_prompt.md for single-shot short video. We strongly recommend running your input through the matching enhancer before inference; un-enhanced prompts tend to produce noticeably weaker results.

Create a JSON file under prompts/. Each file is a single object with a prompts list, where every string is one complete shot. A single string produces one shot; multiple strings produce a multi-shot story, with each new shot conditioned on the previous ones through the paired audio-video memory bank.

Inside each string, write these parts in order:

A more convenient prompt-writing workflow will be released as a director agent for everyone to use.

4. Run

python inference.py

Outputs land in inference_result/outputs/<prompt-name>/inference_<timestamp>/.

Hardware

Peak GPU memory is ~46–50 GB at the default 1280 × 736 × 241 frame setting — a single H100/A100 (80 GB) or 48 GB GPU is sufficient. For smaller GPUs, lower resolution or frame count:

python inference.py --num-frames 121 --video-height 480 --video-width 832

Results

Reported Scale

Human Evaluation

GSB user study. Values are the percentage of user preferences.

Acknowledgements

We gratefully acknowledge LTX-Video for the base video generator and Gemma for the text encoder, along with the broader open-source community.

Citation

If Echo-LongVideo helps your research or products, please cite:

@techreport{echo2026longvideo,
  title        = {Echo-LongVideo: Pushing the Frontier of Long Video Generation},
  author       = {{Echo Team @ Joy Future Academy, JD}},
  institution  = {Joy Future Academy, JD},
  year         = {2026},
  month        = {June},
  url          = {https://github.com/Echo-Team-Joy-Future-Academy-JD/JoyAI-Echo}
}

License

Released under the LTX-2 Community License Agreement. By downloading or using these weights, you agree to its terms. The bundled Gemma text encoder is governed by Google's separate Gemma license.