Klear-46B-A2.5B-Base

Klear

<div align="center"> <img src="figures/klear-logo-02.png" width="500"/> <p> 🤗 <a href="https://huggingface.co/Kwai-Klear">Hugging Face</a> | 💻 <a href="https://github.com/Kwai-Klear/Klear1.0/tree/main">Github Repository</a> | 📑 <a href="https://huggingface.co/Kwai-Klear/Klear-46B-A2.5B-Base">Technique Report</a> | 💬 <a href="https://github.com/Kwai-Klear/Klear1.0/issues">Issues & Discussions</a> </p> </div>

🔥News

• 2025.09.05: We’ve released the Klear-46B-A2.5B series, which currently includes a base model and an instruction-tuned model with DPO. A reasoning-enhanced variant is also in training — stay tuned for upcoming updates!

1. Introduction

Klear-46B-A2.5B is a sparse Mixture-of-Experts (MoE) large language model developed by the Kwai-Klear Team at Kuaishou, designed to deliver both high performance and inference efficiency. It features 256 experts, with only 8 experts and 1 shared expert activated per layer during the forward pass, resulting in 46 billion total parameters but just 2.5 billion active — achieving dense-level performance at a fraction of the computational cost.

The model was trained on over 22 trillion tokens using a three-stage progressive curriculum:

1. Foundational Knowledge Learning (12T tokens): General-purpose datasets such as CommonCrawl were processed with stratified quality filters, following a curriculum learning strategy that progresses from lower to higher data quality.

2. Data Complexity Enhancement (8T tokens): The proportion of mathematical, coding, and STEM-related data was gradually increased to strengthen the model's reasoning and problem-solving capabilities.

3. Reasoning Enhancement and Longcontext Stage (2T tokens): Training focused on synthetic and reasoning-intensive data, combined with a fast learning rate annealing strategy to maximize data efficiency and optimize final performance.

As a result, Klear-46B-A2.5B-Base matches or surpasses the performance of dense models with several times more active parameters, while offering significantly better efficiency and cost-effectiveness for real-world deployment.

Model Summary

The base and instruction tuned + DPO models have the following architecture:

Model Downloads

<div align="center"> </div>

2. Benchmark Evaluation

Klear-46B-A2.5B-Base Evaluation Results

Note:

1. Results marked with * are sourced from their public report, other evaluations are conducted based on internal evaluation frameworks.
2. †During pretraining, we found that the HumanEval metric fluctuated significantly and was extremely sensitive to formatting. Therefore, we referred to the prompt from Ling-series paper to modify the original HumanEval. The results in the table are the evaluation metrics after this modification.

Klear-46B-A2.5B-Instruct Evaluation Results

Note:

1. For InternLM3-8B-Instruct, the results marked with * are sourced from their official website, other evaluations are conducted based on internal evaluation frameworks.
2. For Multi-IF, we report the overall average computed across all three rounds, pooling the Chinese and English metrics.

3. Quick start

Inference with huggingface

You can now inference in Transformers starting from version 4.56.0 and set trust_remote_code=True.

Klear-46B-A2.5B-Base

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_path = "/path/to/Klear-Base"
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
    
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto", dtype=torch.bfloat16, trust_remote_code=True)

text = "世界上最大的湖是"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs.to(model.device), max_new_tokens=256)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

Klear-46B-A2.5B-Instruct

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig

model_path = "/path/to/Klear-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_path)

model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto", dtype=torch.bfloat16, trust_remote_code=True)

messages = [
    {"role": "user", "content": "帮我用 python 写一个计算器的代码吧。"}
]
input_tensor = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(input_tensor.to(model.device), max_new_tokens=1024)

result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True)
print(result)

Inference with vllm

vLLM is a high-speed and memery-efficicent inference framework. We provide our own forked version of vLLM here.

git clone https://github.com/Kwai-Klear/vllm.git
cd vllm
VLLM_USE_PRECOMPILED=1 pip install --editable .
vllm serve /path/to/Klear-Instruct --port 8000 --tensor-parallel-size 8 --trust-remote-code

An OpenAI-compatible API will be available at http://localhost:8000/v1.

Or you can refer to the following Python script for offline inference

import torch
from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

model_path = "/path/to/Klear-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

llm = LLM(
    model=model_path,
    trust_remote_code=True,
    tensor_parallel_size=torch.cuda.device_count(),
    gpu_memory_utilization=0.7
)
messages = [
    {"role": "user", "content": "帮我用 python 写一个计算器的代码吧。"}
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

sampling_params = SamplingParams(
    temperature=0.6, top_p=0.95, top_k=40, max_new_tokens=1024
)

outputs = llm.generate([prompt], sampling_params)

print(outputs[0].outputs[0].text)