ngut-32k

NGUT

Special Tokens

Core

<pad>, <bos>, <eos>, <unk>, <mask>

Conversation

<|turn>, <turn|>, <|document>, <document|>, <|segment>, <segment|>, <|literal>, <literal|>, <|separator|>

Reasoning

<|think|>, <|channel>, <channel|>

Tools

<|tool>, <tool|>, <|tool_call>, <tool_call|>, <|tool_response>, <tool_response|>, <|"|>, <|call_id>, <call_id|>, <|tool_error>, <tool_error|>, <|tool_approval>, <tool_approval|>

Modalities

<|image>, <image|>, <|image|>, <|audio>, <audio|>, <|audio|>, <|video>, <video|>, <|video|>, <|file>, <file|>, <|file|>, <|image_pad|>, <|audio_pad|>, <|video_pad|>, <|modality_separator|>

Speech and Audio

<|transcript>, <transcript|>, <|translation>, <translation|>, <|language>, <language|>, <|speaker>, <speaker|>, <|timestamp>, <timestamp|>, <|audio_event>, <audio_event|>, <|emotion>, <emotion|>, <|music>, <music|>, <|pitch>, <pitch|>, <|velocity>, <velocity|>, <|transcribe|>, <|translate|>, <|caption|>, <|keyword|>, <|timestamps|>, <|no_timestamps|>, <|silence|>

Vision

<|object_ref>, <object_ref|>, <|box>, <box|>, <|quad>, <quad|>, <|point>, <point|>, <|region>, <region|>, <|image_row|>

Code

<|fim_prefix|>, <|fim_middle|>, <|fim_suffix|>, <|fim_pad|>, <|code>, <code|>, <|json>, <json|>

Retrieval

<|source>, <source|>, <|citation>, <citation|>, <|quote>, <quote|>, <|context>, <context|>, <|memory>, <memory|>

Chat Template

<bos><|turn>system
You are a helpful assistant.<turn|>
<|turn>user
Hello!<turn|>
<|turn>model
Hi!<turn|>