CogView4-6B

CogView4-6B

<p style="text-align: center;"> <div align="center"> <img src=https://github.com/THUDM/CogView4/raw/main/resources/logo.svg width="50%"/> </div> <p align="center"> <a href="https://huggingface.co/spaces/THUDM-HF-SPACE/CogView4">🤗 Space | </a> <a href="https://github.com/THUDM/CogView4">🌐 Github </a> | <a href="https://arxiv.org/pdf/2403.05121">📜 CogView3 Paper </a> </p>

Inference Requirements and Model Introduction

• Resolution: Width and height must be between 512px and 2048px, divisible by 32, and ensure the maximum number of pixels does not exceed 2^21 px.
• Precision: BF16 / FP32 (FP16 is not supported as it will cause overflow resulting in completely black images)

Using BF16 precision with batchsize=4 for testing, the memory usage is shown in the table below:

Quick Start

First, ensure you install the diffusers library from source.

pip install git+https://github.com/huggingface/diffusers.git
cd diffusers
pip install -e .

Then, run the following code:

from diffusers import CogView4Pipeline

pipe = CogView4Pipeline.from_pretrained("THUDM/CogView4-6B", torch_dtype=torch.bfloat16)

# Open it for reduce GPU memory usage
pipe.enable_model_cpu_offload()
pipe.vae.enable_slicing()
pipe.vae.enable_tiling()

prompt = "A vibrant cherry red sports car sits proudly under the gleaming sun, its polished exterior smooth and flawless, casting a mirror-like reflection. The car features a low, aerodynamic body, angular headlights that gaze forward like predatory eyes, and a set of black, high-gloss racing rims that contrast starkly with the red. A subtle hint of chrome embellishes the grille and exhaust, while the tinted windows suggest a luxurious and private interior. The scene conveys a sense of speed and elegance, the car appearing as if it's about to burst into a sprint along a coastal road, with the ocean's azure waves crashing in the background."
image = pipe(
    prompt=prompt,
    guidance_scale=3.5,
    num_images_per_prompt=1,
    num_inference_steps=50,
    width=1024,
    height=1024,
).images[0]

image.save("cogview4.png")

Model Metrics

We've tested on multiple benchmarks and achieved the following scores:

DPG-Bench

GenEval

T2I-CompBench

Chinese Text Accuracy Evaluation

Citation

🌟 If you find our work helpful, please consider citing our paper and leaving valuable stars

@article{zheng2024cogview3,
  title={Cogview3: Finer and faster text-to-image generation via relay diffusion},
  author={Zheng, Wendi and Teng, Jiayan and Yang, Zhuoyi and Wang, Weihan and Chen, Jidong and Gu, Xiaotao and Dong, Yuxiao and Ding, Ming and Tang, Jie},
  journal={arXiv preprint arXiv:2403.05121},
  year={2024}
}

License

This model is released under the Apache 2.0 License.