VibeThinker-3B-litert-lm

VibeThinker-3B LiteRT-LM

This repository contains LiteRT-LM conversions of WeiboAI/VibeThinker-3B for local on-device inference.

VibeThinker-3B is a 3B-parameter dense reasoning model built on Qwen2.5-Coder-3B. These artifacts were exported from the Hugging Face safetensors checkpoint with LiteRT Torch and packaged as .litertlm files for the LiteRT-LM runtime.

Files

The CPU/GPU .litertlm files include a compressed Hugging Face tokenizer, LLM metadata, a quantized prefill/decode TFLite model, and a quantized external embedder.

Run With LiteRT-LM

Install the LiteRT-LM CLI:

uv tool install litert-lm

Run the generic artifact:

litert-lm run \
  --from-huggingface-repo Tdamre/VibeThinker-3B-litert-lm \
  VibeThinker-3B.litertlm \
  --backend=cpu \
  --prompt="What is 17 * 3? Answer with just the number."

Run the lower-cache artifact:

litert-lm run \
  --from-huggingface-repo Tdamre/VibeThinker-3B-litert-lm \
  VibeThinker-3B-web.litertlm \
  --backend=cpu \
  --prompt="What is 2+2? Answer with just the number."

Run the 32k-context artifact:

litert-lm run \
  --from-huggingface-repo Tdamre/VibeThinker-3B-litert-lm \
  VibeThinker-3B-c32k.litertlm \
  --backend=cpu \
  --prompt="What is 17 * 3? Answer with just the number."

Run the 64k-context artifact:

litert-lm run \
  --from-huggingface-repo Tdamre/VibeThinker-3B-litert-lm \
  VibeThinker-3B-c64k.litertlm \
  --backend=cpu \
  --prompt="What is 2+2? Answer with just the number."

Conversion Summary

Source model revision:

WeiboAI/VibeThinker-3B@0c7115fdd0957b3da0f2a0829ab1763969d30300

CPU/GPU conversion command pattern:

litert-torch export_hf \
  model-cache/WeiboAI-VibeThinker-3B \
  <output_dir> \
  --keep_temporary_files=True \
  --prefill_lengths=128,1024 \
  --cache_length=<2048-or-4096-or-32768-or-65536> \
  --externalize_embedder=True \
  --quantization_recipe=dynamic_wi8_afp32

Toolchain:

Python 3.12.12
litert-torch 0.9.1
litert-lm 0.13.1
ai-edge-litert 2.1.5
ai-edge-quantizer 0.7.0
torch 2.12.0+cu130
transformers 5.9.0

Not Included

• Qualcomm SM8750 NPU AOT artifacts were not produced in this initial pass.
• MediaPipe .task bundles were not uploaded because VibeThinker-3B ships an HF tokenizer.json rather than a SentencePiece tokenizer.model.