mario-videogamegen

MarioVGG - Video Game Generation: A Practical Study using Mario

Virtuals Protocol

Visit the project website

Read the paper

Abstract

This work reports the study of video generative models for interactive video game generation and simulation. We discuss and explore the use of available pre-trained open-sourced video generation models to create playable interactive video games. While being able to generate short clips of broad range of described scenes, such models still lack controllability and continuity. Given these limitations, we focus on producing and demonstrating a reliable and controllable video game generator on a single game domain. We present MarioVGG, a text-to-video diffusion model for controllable video generation on the Super Mario Bros game. MarioVGG demonstrates the ability to continuously generate consistent and meaningful scenes and levels, as well as simulate the physics and movements of a controllable player all through video.

BibTex

@misc{virtuals2024videogame,
  title={Video Game Generation: A Practical Study Using Mario},
  author={Virtuals Protocol},
  note={Preprint},
  url={https://github.com/Virtual-Protocol/mario-videogamegen/blob/main/static/pdfs/VideoGameGen.pdf},
  year={2024}
}