ConsistCompose-BAGEL-7B-MoT

EN | 中文

ConsistCompose: Unified Multimodal Layout Control for Image Composition

<a href="https://github.com/OpenSenseNova/ConsistCompose"><img alt="GitHub Stars" src="https://img.shields.io/github/stars/OpenSenseNova/ConsistCompose?style=social"></a> <a href="https://OpenSenseNova.github.io/ConsistCompose/"><img alt="Project Page" src="https://img.shields.io/badge/Project%20Page-ConsistCompose-blue"></a> <a href="https://arxiv.org/abs/2511.18333"><img alt="arXiv" src="https://img.shields.io/badge/arXiv-2511.18333-b31b1b.svg"></a> <a href="https://huggingface.co/sensenova/ConsistCompose"><img src="https://img.shields.io/static/v1?label=%F0%9F%A4%97%20Hugging%20Face&message=Model&color=green"></a> <a href="https://huggingface.co/datasets/sensenova/ConsistCompose3M"><img src="https://img.shields.io/static/v1?label=%F0%9F%A4%97%20Hugging%20Face&message=Dataset&color=yellow"></a>

Overview

Despite significant advancements in unified multimodal models that integrate visual understanding and image generation, precise layout control for multi-instance image synthesis remains an underexplored challenge. Existing approaches often rely on task-specific modules or structured spatial encoders, limiting their compatibility with broader multimodal capabilities like visual reasoning and identity preservation.

In this work, we propose ConsistCompose, a unified multimodal framework designed to enable layout-controllable multi-instance generation through a language-driven paradigm. Built upon the MoT architecture of Bagel, ConsistCompose introduces Linguistic-Embedded Layout-Grounded Generation (LELG), which embeds layout coordinates directly into language prompts as textual tokens—eliminating the need for specialized spatial encoders or geometric branches. To support large-scale training, we construct ConsistCompose3M, a 3.4M-sample dataset with layout and identity annotations, including 2.6M text-guided and 0.8M image-guided pairs that provide structured spatial and semantic supervision.

ConsistCompose achieves state-of-the-art performance on layout-controlled generation benchmarks: it delivers a 7.2% gain in layout IoU and a 13.7% AP improvement on COCO-Position (with 92.6% Instance Success Ratio and 76.1% Image Success Ratio), while leading on core metrics (DINO, mIoU, AP) across both MS-Bench and MS-Bench-Random. Notably, the framework preserves strong general multimodal capabilities—maintaining performance on MMBench, MMMU, and GenEval comparable to its Bagel backbone—while enhancing identity preservation for multi-reference generation. A key component, Coordinate-CFG, further strengthens spatial fidelity through hierarchical classifier-free guidance, balancing strict layout adherence with visual realism.

More importantly, ConsistCompose establishes a unified paradigm that consolidates layout-grounded text-to-image synthesis, multi-reference identity-preserving composition, and general multimodal understanding within a single generative interface. We analyze the impact of coordinate guidance scaling, validate the flexibility of the LELG paradigm, and demonstrate its ability to handle complex, cluttered layouts without sacrificing semantic coherence. ConsistCompose provides a principled solution to layout-controllable multimodal generation, pointing toward future directions in finer-grained spatial reasoning and interactive scene composition.

All technical details, training recipes, and supplementary results are publicly available to facilitate further research in unified layout-aware multimodal systems.

Release Information

We introduce ConsistCompose-BAGEL-7B-MoT, built upon a unified understanding and generation base model.

Specific details are reported in the ConsistCompose technical report, Our ConsistCompose-BAGEL-7B-MoT preserves the full image general generation and understanding capabilities of the original BAGEL while introducing enhanced layout control for precise multi-instance composition.

1 Results on COCO-Position

<table> <thead> <tr> <th rowspan="2">Methods</th> <th colspan="6">Instance Success Ratio (%) ↑</th> <th colspan="6">Image Success Ratio (%) ↑</th> <th colspan="4">Position Accuracy (%) ↑</th> </tr> <tr> <th>L2</th><th>L3</th><th>L4</th><th>L5</th><th>L6</th><th>Avg</th> <th>L2</th><th>L3</th><th>L4</th><th>L5</th><th>L6</th><th>Avg</th> <th>mIoU</th><th>AP</th><th>AP50</th><th>AP75</th> </tr> </thead> <tbody> <tr> <td>GLIGEN</td> <td>89.1</td><td>86.3</td><td>82.0</td><td>79.6</td><td>81.6</td><td>82.6</td> <td>78.8</td><td>63.8</td><td>48.1</td><td>35.0</td><td>35.0</td><td>52.1</td> <td>69.0</td><td>40.5</td><td>75.9</td><td>39.1</td> </tr> <tr> <td>InstanceDiffusion</td> <td><u>94.1</u></td><td><b>94.4</b></td><td><u>89.5</u></td><td><u>84.6</u></td><td><u>83.8</u></td><td><u>87.8</u></td> <td><u>89.4</u></td><td><b>84.4</b></td><td><u>67.5</u></td><td>46.9</td><td><u>39.4</u></td><td><u>65.5</u></td> <td><u>78.1</u></td><td><u>57.2</u></td><td><u>83.6</u></td><td><u>65.5</u></td> </tr> <tr> <td>MIGC++</td> <td><u>94.1</u></td><td>92.1</td><td>87.3</td><td>84.1</td><td>83.4</td><td>86.8</td> <td><u>89.4</u></td><td>78.1</td><td>62.5</td><td><u>48.1</u></td><td>38.8</td><td>63.4</td> <td>74.9</td><td>48.3</td><td>79.2</td><td>52.6</td> </tr> <tr> <td>CreatiLayout</td> <td>81.9</td><td>76.3</td><td>73.4</td><td>73.5</td><td>71.2</td><td>74.0</td> <td>69.4</td><td>48.1</td><td>36.9</td><td>31.9</td><td>26.3</td><td>42.5</td> <td>64.9</td><td>32.4</td><td>61.1</td><td>31.6</td> </tr> <tr> <td>PlanGen</td> <td>85.3</td><td>84.2</td><td>83.8</td><td>80.9</td><td>81.2</td><td>82.5</td> <td>72.5</td><td>63.1</td><td>51.3</td><td>33.1</td><td>31.3</td><td>50.3</td> <td>66.2</td><td>31.9</td><td>74.0</td><td>21.5</td> </tr> <tr> <td><b>Ours</b></td> <td><b>95.6</b></td><td><u>94.2</u></td><td><b>92.7</b></td><td><b>90.6</b></td><td><b>92.4</b></td><td><b>92.6</b></td> <td><b>91.9</b></td><td><u>83.1</u></td><td><b>73.1</b></td><td><b>63.7</b></td><td><b>68.8</b></td><td><b>76.1</b></td> <td><b>85.3</b></td><td><b>70.9</b></td><td><b>89.1</b></td><td><b>76.9</b></td> </tr> </tbody> </table>

2 Results on MS-Bench && MS-Bench-Random

<table> <thead> <tr> <th rowspan="2">Methods</th> <th colspan="4">MS-Bench</th> <th colspan="4">MS-Bench-Random</th> </tr> <tr> <th>CLIP-T</th><th>DINO</th><th>mIoU</th><th>AP</th> <th>CLIP-T</th><th>DINO</th><th>mIoU</th><th>AP</th> </tr> </thead> <tbody> <tr> <td>GLIGEN</td> <td>0.309</td><td>0.454</td><td><u>0.868</u></td><td><u>0.751</u></td> <td>0.312</td><td>0.431</td><td><u>0.858</u></td><td><u>0.722</u></td> </tr> <tr> <td>MS-Diffusion</td> <td><b>0.336</b></td><td>0.555</td><td>0.466</td><td>0.108</td> <td><u>0.334</u></td><td>0.544</td><td>0.464</td><td>0.105</td> </tr> <tr> <td>MUSE</td> <td>0.320</td><td><u>0.619</u></td><td>0.698</td><td>0.352</td> <td>0.321</td><td><u>0.607</u></td><td>0.673</td><td>0.303</td> </tr> <tr> <td><b>Ours</b></td> <td><u>0.333</u></td><td><b>0.660</b></td><td><b>0.889</b></td><td><b>0.789</b></td> <td><b>0.334</b></td><td><b>0.630</b></td><td><b>0.878</b></td><td><b>0.756</b></td> </tr> </tbody> </table>

3. Results on MMBench，MMMU, GenEval, GEdit

<table> <thead> <tr> <th colspan="5">(a) General capability</th> </tr> <tr> <th>Model</th> <th>MMBench↑</th> <th>MMMU↑</th> <th>GenEval↑</th> <th>GEdit↑</th> </tr> </thead> <tbody> <tr> <td>Bagel Base</td> <td>81.4</td> <td>46.4</td> <td>0.86</td> <td>6.68</td> </tr> <tr> <td>Ours (w/o Coord)</td> <td>81.5</td> <td>39.4</td> <td>0.88</td> <td>6.23</td> </tr> <tr> <td>Ours (w/ Coord)</td> <td>81.4</td> <td>42.3</td> <td>0.88</td> <td>6.31</td> </tr> </tbody> </table>

4. Results on DreamBench(Single/Multi)

<table> <thead> <tr> <th rowspan="2">Method</th> <th colspan="3">Single</th> <th colspan="3">Multi</th> </tr> <tr> <th>DINO</th> <th>CLIP-I</th> <th>CLIP-T</th> <th>DINO</th> <th>CLIP-I</th> <th>CLIP-T</th> </tr> </thead> <tbody> <tr> <td>UNO</td> <td>0.661</td> <td>0.796</td> <td>0.304</td> <td>0.491</td> <td>0.715</td> <td>0.323</td> </tr> <tr> <td>OmniGen</td> <td>0.554</td> <td>0.746</td> <td>0.322</td> <td>0.441</td> <td>0.692</td> <td>0.341</td> </tr> <tr> <td>OmniGen2</td> <td>0.671</td> <td>0.791</td> <td>0.312</td> <td>0.459</td> <td>0.698</td> <td>0.333</td> </tr> <tr> <td><b>Ours</b></td> <td>0.677</td> <td>0.792</td> <td>0.314</td> <td>0.506</td> <td>0.703</td> <td>0.335</td> </tr> </tbody> </table>

🖊️ Citation

@InProceedings{Shi_2026_CVPR,
    author    = {Shi, Xuanke and Li, Boxuan and Han, Xiaoyang and Cai, Zhongang and Yang, Lei and Wang, Quan and Lin, Dahua},
    title     = {ConsistCompose: Unified Multimodal Layout Control for Image Composition},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2026},
    pages     = {495-505}
}