Youtu-LLM-A7B-Base

Youtu-LLM-A7B-Base

Youtu-LLM-A7B-Base is a pre-trained checkpoint of a vision-language Mixture-of-Experts model with about 120B total parameters and ~7B activated parameters per token.

• This is a base model. We recommend to use raw text continuation, although chat_template.jinja is provided for further purpose. Custom modeling code is shipped in this repository and must be loaded with trust_remote_code=True.
• It combines a hybrid attention stack, KDA interleaved with GatedMLA, with a fine-grained MoE feed-forward network. The model accepts text and image inputs and is trained with a 262,144-token context window over 11.5T multimodal tokens.
• Compared to typical MLLMs, this model employs an opposed pre-training strategy: it attaches a randomly initialized LLM to a pre-trained ViT, followed by a multi-stage pre-training process with context lengths of 4K, 32K, 128K, and 256K.

Model Summary

Evaluation Results

Text

Vision-Language

For VL evaluation, MCQ-style benches report option-letter EM after extraction. Open-ended benches use MiniMax-M2.7 for MLLM-as-a-judge. Other columns are the official suite metrics (not generic accuracy).

Transformers

Install transformers==5.12.1, plus torch, accelerate and pillow for images.

pip install transformers==5.12.1 torch accelerate pillow

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_path = "tencent/Youtu-LLM-A7B-Base"

processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

prompt = "The answer of Q1 is (A). The answer of Q2 is (B). The answer of Q3 is ("
inputs = processor(text=[prompt], return_tensors="pt")
inputs = {k: v.to(model.device) if hasattr(v, "to") else v for k, v in inputs.items()}

outputs = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(processor.tokenizer.decode(outputs[0], skip_special_tokens=False))

For an image, pass images=[pil_image] to the processor together with a prompt that contains <|vision_start|><|image_pad|><|vision_end|>.

SGLang

Install sglang==0.5.17 and start the server with the adapter shipped in this repository (sglang_adapt/launch.py), not python -m sglang.launch_server.

cd sglang_adapt
MODEL_PATH=tencent/Youtu-LLM-A7B-Base \
  TP_SIZE=4 DP_SIZE=2 EP_SIZE=4 PORT=8000 \
  bash serve_youtu_vl_moe_sglang.sh

curl http://127.0.0.1:8000/generate \
  -H 'Content-Type: application/json' \
  -d '{"text":"The answer of Q1 is (A). The answer of Q2 is (B). The answer of Q3 is (","sampling_params":{"temperature":0,"max_new_tokens":16}}'

Note

• Use a Python environment that already has SGLang on PATH, or set SGLANG_ENV.
• Weights are BF16; the serve script sets --dtype bfloat16.
• On 8 GPUs, the recommended layout is TP_SIZE=4, DP_SIZE=2, EP_SIZE=4.
• This is a base model: it is recommended to send raw prompts to POST /generate, although chat_template.jinja is provided for further purpose.

Overlap scheduler

DISABLE_OVERLAP_SCHEDULE defaults to 1 (off). For text-only serving you can enable it for higher decode throughput:

DISABLE_OVERLAP_SCHEDULE=0 MODEL_PATH=tencent/Youtu-LLM-A7B-Base \
  TP_SIZE=4 DP_SIZE=2 EP_SIZE=4 bash serve_youtu_vl_moe_sglang.sh

Do not enable it for high-concurrency image traffic — vision prefill can hang in event_loop_overlap.

License

See LICENSE.txt.