MiniVeo3-Reasoner-Maze-5B

<p align="center"><img src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/assets/miniveo3-reasoner-logo-pure.png" width="150px" alt="MiniVeo3-Reasoner icon" /></p> <h1 align="center"> MiniVeo3-Reasoner: Thinking with Videos from Open-Source Priors </h1> <p align="center"> <a href="https://github.com/thuml/MiniVeo3-Reasoner"><img src="https://img.shields.io/badge/GitHub-000000?style=for-the-badge&logo=github&logoColor=white" alt="GitHub"></a> <a href="https://huggingface.co/thuml/MiniVeo3-Reasoner-Maze-5B"><img src="https://img.shields.io/badge/🤗_HuggingFace-fcd022?style=for-the-badge" alt="HuggingFace"></a> </p>

🎯 Overview

Advanced video models have recently demonstrated remarkable zero-shot capabilities of visual reasoning, solving tasks like maze, symmetry, and analogy completion through a chain-of-frames (CoF) reasoning process.

This project shows that such CoF capability can be acquired by fine-tuning open-source video models like Wan2.2.

In the maze domain, the fine-tuned models—dubbed MiniVeo3-Reasoner—exhibit surprisingly strong visual reasoning performance, achieving near-perfect accuracy on in-distribution tests and robust out-of-distribution generalization.

Under controlled comparisons, MiniVeo3-Reasoner significantly outperforms baseline approaches that reason in other modalities such as text or images.

We further envision that this visual reasoning capability can be enhanced through reinforcement learning of video models.

🔥 News

• 🚩 2025.10: We are thrilled to release MiniVeo3-Reasoner, with mazes as a testbed for visual reasoning!

🤗 Models

✨ Examples

<table style="width: 100%; text-align: center; margin-top: 20px;"> <tr> <td> <b>Problem Setup</b></td> <td colspan=2> <b>Examples</b></td> </tr> <tr> <td>Maze 3x3</td> <td > <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze3_1.mp4" width="80%" controls autoplay loop></video> </td> <td> <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze3_2.mp4" width="80%" controls autoplay loop></video> </td> </tr> <tr> <td>Maze 4x4</td> <td > <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze4_1.mp4" width="80%" controls autoplay loop></video> </td> <td> <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze4_2.mp4" width="80%" controls autoplay loop></video> </td> </tr> <tr> <td>Maze 5x5</td> <td > <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze5_1.mp4" width="80%" controls autoplay loop></video> </td> <td> <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze5_2.mp4" width="80%" controls autoplay loop></video> </td> </tr> <tr> <td>Maze 6x6</td> <td > <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze6_1.mp4" width="80%" controls autoplay loop></video> </td> <td> <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze6_2.mp4" width="80%" controls autoplay loop></video> </td> </tr> </table>

OOD Generalization

OOD Solution Lengths:

<table style="width: 100%; text-align: center; margin-top: 20px;"> <tr> <td> <b>Problem Setup</b></td> <td colspan=2> <b>Examples</b></td> </tr> <tr> <td>Maze 6x6 <br/> (solution len > 12)</td> <td > <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze6ood_1.mp4" width="80%" controls autoplay loop></video> </td> <td> <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze6ood_2.mp4" width="80%" controls autoplay loop></video> </td> </tr> </table>

OOD Maze Sizes:

<table style="width: 100%; text-align: center; margin-top: 20px;"> <tr> <td> <b>Problem Setup</b></td> <td colspan=2> <b>Examples</b></td> </tr> <tr> <td>Maze 7x7</td> <td > <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze7_1.mp4" width="80%" controls autoplay loop></video> </td> <td> <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze7_2.mp4" width="80%" controls autoplay loop></video> </td> </tr> <tr> <td>Maze 8x8</td> <td > <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze8_1.mp4" width="80%" controls autoplay loop></video> </td> <td> <video src="https://github.com/thuml/MiniVeo3-Reasoner/raw/main/showcases/maze/maze8_2.mp4" width="80%" controls autoplay loop></video> </td> </tr> </table>

📊 Performance

Following Visual Planning: Let's Think Only with Images, we report two metrics:

• Exact Match (EM) measures whether the model successfully generates the complete and correct trajectory that aligns with the shortest optimal valid path.
• Progress Rate (PR) measures the number of consecutively correct steps (valid forward moves) from the start to the number of steps in the optimal path.

Comparisons

Under the same amount of training data, we include performance metrics reported in Visual Planning for reference and comparison.

🚀 Get Started

Environment Setup

conda create -n miniveo3_reasoner python==3.12
conda activate miniveo3_reasoner
pip install -r requirements.txt

We use DiffSynth-Studio for diffusion model training and inference. You need also install it:

git clone https://github.com/modelscope/DiffSynth-Studio.git
cd DiffSynth-Studio
git checkout ed256ef8be195d5deae2846a7e9f025670d99db3
pip install -e .

Data Preparation

Our data generator produces a series of mazes with configurable size, path length and amount, outputting a .mp4 video file and a .png image (the first frame of the video).

We use a customized version of maze-dataset. You can install it as follows:

pip install -e data/maze/maze-dataset

After installation, use the script below to generate mazes with custom configurations:

python data/maze/maze_generator.py

To reproduce the same data distribution used in our experiments, simply run:

bash scripts/generate_maze_dataset.sh

The result will be in dataset/maze_train and dataset/maze_test respectively.

Inference

Download our LoRA weights:

pip install "huggingface_hub[cli]"
huggingface-cli download thuml/MiniVeo3-Reasoner-Maze-5B --local-dir models/thuml/MiniVeo3-Reasoner-Maze-5B

To run inference on a single file or directory, use:

python inference/maze/inference_maze.py [-r] filename_or_directory

💡 The first run may take additional time to automatically download the base model files.

To perform inference on all test samples, simply run:

bash scripts/inference_maze_testset.sh

Success Evaluation

Our evaluator compares the predicted trajectory with the ground truth, computing the distance between the two paths.

We implement our own versions of Exact Match (EM) and Progress Rate (PR) metrics for video-based evaluation.

If your generated results are stored in dataset/maze_test and named properly, you can evaluate all test samples by running:

bash scripts/evaluate_maze.sh

Training Models

We train Wan2.2-TI2V-5B with LoRA, following the instructions provided in DiffSynth-Studio. You can easily fine-tune your own models using the same framework.

For your convenience, if you follow ours, you can copy the train dataset dataset/maze_train directly into DiffSynth-Studio/data/example_video_dataset.

🤝 Contributors

Jialong Wu*, Tianhao Huang*, Changjing He*, Mingsheng Long. (*: Equal Contribution)

We welcome contributions! Feel free to open GitHub issues for bug reports or feature requests.

💡 Acknowledgements

• Veo 3: This project is inspired by the impressive zero-shot performance of Veo 3!
• Wan: Powerful open-source video diffusion models used as base models.
• DiffSynth-Studio: Video diffusion model training.
• maze-dataset: Data generation for maze reasoning tasks.
• Visual Planning: Baseline benchmark for performance comparison.
• Nano Banana: Help in generating the project logo.

📜 Citation

There is currently no technical report available.

If you find MiniVeo3-Reasoner useful, we would appreciate it if you could cite our work:

@misc{miniveo3reasoner,
    title = {MiniVeo3-Reasoner: Thinking with Videos from Open-Source Priors},
    author = {Jialong Wu, Tianhao Huang, Changjing He, Mingsheng Long},
    year = {2025},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{https://github.com/thuml/MiniVeo3-Reasoner}},
}