UniGen-LingXi-5B

UniGen-LingXi: A Unified Multi-Modal Generation and Editing Framework

<p align="center">

简体中文 | English

</p>

LingXi Qihang Presents — Bridging AI and Creativity with Intuition

UniGen-LingXi is a unified inference framework built upon Kiwi-Edit-5B, supporting nine generation and editing tasks across both image and video modalities. One model, one framework, one unified API.

📖 Overview

We present UniGen-LingXi, a unified framework that extends a single video editing model (Kiwi-Edit) to perform nine distinct generation and editing tasks. By carefully constructing conditional videos and task-specific prompts, we achieve this without any architectural modification. Our data construction pipeline and memory-efficient training script make training feasible on consumer-grade GPUs.

---

📋 Nine Supported Tasks

Task Matrix: Output Modality × Condition Type

---

🚀 Quick Start

1. Environment Setup

# Install dependencies
pip install -r requirements.txt

2. Download the Model

The model is released on HuggingFace and ModelScope:

• HuggingFace: shyai/UniGen-LingXi-5B
• ModelScope: haohanxingcheng/UniGen-LingXi-5B

3. Usage Examples

t2i — Text-to-Image

python diffusers_Uni_Gen.py \
    --task_type text_to_image \
    --prompt "text-to-image: A beautiful sunset over the ocean" \
    --save_path output/t2i.png \
    --model_path <path_to_model>

i2i — Image Editing

python diffusers_Uni_Gen.py \
    --task_type image_edit \
    --src_image input.jpg \
    --prompt "Image-editing: Turn the sky into sunset" \
    --save_path output/edited.png \
    --model_path <path_to_model>

t2v — Text-to-Video

python diffusers_Uni_Gen.py \
    --task_type text_to_video \
    --prompt "text-to-video: A cat running on grass" \
    --save_path output/t2v.mp4 \
    --model_path <path_to_model>

i2v — Image-to-Video

python diffusers_Uni_Gen.py \
    --task_type image_to_video \
    --src_image input.jpg \
    --prompt "image-to-video: Make the flower bloom" \
    --save_path output/i2v.mp4 \
    --model_path <path_to_model>

VE — Video Editing

python diffusers_Uni_Gen.py \
    --task_type video_edit \
    --video_path input.mp4 \
    --prompt "video-editing: Convert the video into an oil painting style" \
    --save_path output/ve.mp4 \
    --model_path <path_to_model>

Ref-IE — Reference-Guided Image Editing

python diffusers_Uni_Gen.py \
    --task_type image_edit \
    --src_image source.jpg \
    --ref_image reference.jpg \
    --prompt "Image-editing: Apply the style from reference image" \
    --save_path output/ref_ie.png \
    --model_path <path_to_model>

Ref-T2I — Reference-Guided Text-to-Image

python diffusers_Uni_Gen.py \
    --task_type text_to_image \
    --ref_image reference.jpg \
    --prompt "text-to-image: Generate an image in the style of reference" \
    --save_path output/ref_t2i.png \
    --model_path <path_to_model>

Ref-VE — Reference-Guided Video Editing

python diffusers_Uni_Gen.py \
    --task_type video_edit \
    --video_path input.mp4 \
    --ref_image style_ref.jpg \
    --prompt "video-editing: Transform video to match the reference style" \
    --save_path output/ref_ve.mp4 \
    --model_path <path_to_model>

Ref-T2V — Reference-Guided Text-to-Video

python diffusers_Uni_Gen.py \
    --task_type text_to_video \
    --ref_image reference.jpg \
    --prompt "text-to-video: Generate a video with the reference style" \
    --save_path output/ref_t2v.mp4 \
    --model_path <path_to_model>

---

For more detailed usage and qualitative examples for each task, see README_anas.md

Image Editing

Text-to-Image

🛠️ Command-Line Arguments

---

📊 Performance

Image Editing (GEdit-Bench)

Performance Analysis (GEdit-Bench)

Performance Summary

The model exhibits a clear task-specialization pattern in image editing. It excels at local attribute edits such as subject addition, removal, and color alteration (Q_O ≥ 6.7), achieving object-centric modifications with extremely high data efficiency (only 20k samples). Global style transfer and background manipulation reach usable levels (Q_O ≈ 6.2), capable of holistic re-rendering beyond simple filters.

The main limitations are: text editing (1.58) and motion editing (~4.0) perform poorly, rooted in the training data lacking text-rendering and dynamic editing samples, as well as the inherent trade-off in the 9-in-1 architecture between video temporal consistency and pixel-level text precision. The conditioning mechanism works well for global style but is less effective for fine-grained local control. Nevertheless, the model achieves functional validation of 9-in-1 unification under an extremely low data budget (Q_O > 5.0) with competitive perceptual quality (Q_PQ of 6.7), fully demonstrating the efficiency and feasibility of the editing-first unified paradigm.

Strengths: High-precision local control, genuine style reconstruction (not a global filter), adaptive lighting and context integration, strong generalization with only 20k samples.

Weaknesses: Very poor text editing, insufficient motion and portrait retouching, fine-grained local control needs improvement.

Text-to-Image (DPG-Bench)

Performance Analysis (DPG-Bench)

Strengths: Strong spatial and semantic relationship modeling (Relation: 85.58, surpassing multiple open‑source baselines), robust attribute binding (82.32), and reliable object integrity (80.20).

Weaknesses: The "Other" category scores only 50.80, with critical deficits in counting (46.5), text rendering (68.0), and size perception (59.09).

Root Cause: The 20k training samples are primarily from video editing datasets focused on local attribute manipulation, lacking complex composition and explicit reasoning modules; the unified architecture prioritizes structural priors over fine-grained symbolic grounding.

Video Editing (OpenVE-Bench)

Note: Video editing metrics are based on Kiwi-Edit (Stage-3) validation results. The architecture inherits its editing priors with zero performance degradation.

Image Editing (GEdit-Bench): Strong at local edits, weak at text/motion editing.

Video Editing (OpenVE-Bench): Exactly matches the Kiwi-Edit baseline, with editing priors fully preserved.

Text-to-Image (DPG-Bench): Strong at Relation/Attribute, weak at complex composition.

---

🎯 Key Features

• 9-in-1 Capability: A single unified framework supports all nine generation and editing tasks.
• Plug and Play: Diffusers-style API; run with just a few lines of code.
• Efficient Inference: Runs on consumer-grade GPUs.

---

📂 Project Structure

UniGen-LingXi/
├── diffusers_Uni_Gen.py    # Main inference script
├── gradio_UniGen.py        # Gradio Web UI
├── test.sh                 # Nine-task test script
├── assert/                 # Test assets
│   └── images/             # Example outputs
├── README.md               # Project documentation
└── LICENSE

---

🔧 Gradio Web UI

python gradio_UniGen.py

---

⚠️ Important Notes

1. Prompt Format: All prompts must include a task prefix (e.g., text-to-video: , Image-editing: ) to ensure the model correctly interprets the instruction type.
2. Frame Limit: The model supports a maximum of 81 frames.
3. Resolution: 720×1280 or lower is recommended for optimal performance.

🤝 Contact & Feedback

• GitHub Issues: https://github.com/Lingxi-Qihang/UniGen-LingXi/issues
• HuggingFace: https://huggingface.co/shyai/UniGen-LingXi-5B
• ModelScope: https://modelscope.cn/models/haohanxingcheng/UniGen-LingXi-5B

⚠️ Important Disclaimer

• Training code and data construction scripts are not currently open-source: To preserve space for subsequent research and technical iteration, only the inference code and pre-trained model weights are currently released. We welcome technical collaboration and exchange. For commercial or research partnership inquiries, please contact us via email.
• This project is developed based on Kiwi-Edit (Apache-2.0 License), and the original license's copyright notice has been retained as required.

🙏 Acknowledgements

• Kiwi-Edit for providing powerful video/image editing capabilities.
• Thanks to the open-source community and the HuggingFace Diffusers team.

📝 Conclusion

UniGen-LingXi-5B is a resource-efficient, 9-in-1 unified multi-modal generation and editing framework. Grounded in an editing-first philosophy, we reformulate all tasks as conditional video generation and consolidate five core tasks plus four extended reference-guided tasks into a single, unmodified model — trained on a single GPU with only 20k samples. The approach demonstrates exceptional sample efficiency: image editing performance is state-of-the-art, video editing priors are preserved without loss, and generative skills are functionally viable. Through transparent boundary analysis, we reveal a fundamental generality‑precision trade-off and an encoder information bottleneck, providing a clear improvement roadmap for the community. This work not only proves the feasibility of 9‑in‑1 unification but also validates a transferable methodology of editing-first, condition-constructed unification — any stronger DiT backbone or vision‑language model can adopt this paradigm to achieve even broader task unification at minimal adaptation cost.

🗺️ Roadmap

🔬 Near-Term — Targeted Enhancement and Scaling Validation

Directly tackle the identified core bottlenecks — counting reasoning, text rendering, and complex spatial composition — through dedicated data augmentation, numeric reasoning modules, and a dual-encoder architecture for finer conditional control. The training dataset will be scaled to over one hundred thousand samples, rebalancing towards long-tail composition and scene diversity. In parallel, we will explore introducing sparse Mixture-of-Experts (MoE) mechanisms, where task-aware routing dynamically activates different expert sub-networks, fundamentally alleviating the parameter conflict between editing and generation tasks.

💡 Collaborators with compute resources or high-quality data are warmly welcomed to help scale up the model.

🚀 Mid-Term — From Visual Unification to Audio-Visual Joint Generation

The current framework has proven the feasibility of intra-modal visual unification. The next step is to horizontally expand to the audio modality. By introducing an audio encoder as an additional condition, the model will support new tasks such as video-to-audio and audio-to-video, forming a complete multimedia content creation loop.

🌍 Long-Term — Towards World Models and Embodied Intelligence

The editing-first framework's core capability of action preservation and structure retention aligns precisely with the fundamental demand of world models for spatio-temporal consistency. We will progressively introduce physical priors (rigid-body collisions, fluid motion), explicit 3D/4D representations, and causal reasoning capabilities, enabling the model to evolve from "generating visually plausible pixels" to "predicting physically believable futures," ultimately empowering embodied agents to realize the complete closed loop of "language instruction → goal state prediction → action sequence planning."

💡 Researchers interested in world models and embodied intelligence are warmly invited to explore together.

<p align="center">

Bridging AI and Creativity with Intuition 🌟

</p>

---

📄 Citation

If you use UniGen-LingXi in your research, please cite:

@misc{unigenlingxi2026,
  title = {UniGen-LingXi: A Resource-Efficient, Editing-First Framework Unifying 9 Multi-Modal Generation and Editing Tasks},
  author = {Haiying Sha and Yan Zheng},
  year = {2026},
  howpublished = {\url{https://github.com/Lingxi-Qihang/UniGen-LingXi}},
}

Please also cite the underlying model:

@misc{kiwi2026,
  title = {Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance},
  author = {Y. Lin and others},
  year = {2026},
  eprint = {arXiv:2603.02175},
}