bagel-gguf

gguf quantized and fp8/16 scaled version of bagel

• base model from bytedance-seed
• multimodal trial model (i.e., t2i, image editing/recogition)

run it with gguf-connector

ggc b2

<Gallery />

review/reference

• simply execute the command (ggc b2) above in console/terminal
• opt a vae then opt a model file in the current directory to interact (see example below)

Detecting GGUF/Safetensors...

GGUF file(s) available. Select which one for VAE:

1. pig_ae_fp32-f16.gguf
2. pig_ae_fp32-f32.gguf

Enter your choice (1 to 2): 1

VAE file: pig_ae_fp32-f16.gguf is selected!

Safetensors file(s) available. Select which one for MODEL:

1. ema_bf16.safetensors
2. ema_fp16.safetensors (for non-cuda user)
3. ema_fp8_e4m3fn.safetensors (recommended)
4. ema_fp8_e5m2.safetensors

Enter your choice (1 to 4): _

• note: for the latest update, only tokenizer will be pulled to gguf-connector folder (cache) automatically during the first launch; you need to prepare the bulky model and vae files still, working like vision connector right away; mix and match, more flexible
• run it entirely offline; i.e., from local URL: http://127.0.0.1:7860 with lazy webui
• require dependency: bagel2; pip install bagel2; for flash-attn and triton, could opt to install it with pre-built wheels, i.e.,here, unless you can build the wheel yourself successfully
• might need some optional dependencies, please refer to the checklist; as connector won't force your machine to install any of those by default
• gguf-connector (pypi)