granite-4.2-3b-rkllm-rk3576

Granite 4.2 3B RKLLM for RK3576

• Target platform: Rockchip RK3576
• Format: RKLLM
• Weight quantization: W4A16, group size 32
• Context size: 16384

Tested hardware

Tested successfully on Rockchip RK3576 with:

• RKLLM runtime 1.3.0
• RKNPU driver 0.9.8
• 2 NPU cores
• A few sample outputs in all supported languages

Prompt format

The original Granite 4.2 tokenizer and chat template are retained.

Thinking mode uses:

<|im_start|>assistant
<think>

Non-thinking mode uses:

<|im_start|>assistant
<think></think>

Limitations

Unofficial community conversion of ibm-granite/granite-4.2-3b for Rockchip RK3576 using RKLLM Toolkit 1.3.0.

Although the upstream Granite 4.2 model supports a substantially larger context window, this RKLLM build was compiled with a maximum context of 16384 tokens.

Quantization may reduce model quality compared with the original BF16 model.

This build is specific to the Rockchip RK3576 RKLLM runtime and is not a GGUF or standard Transformers checkpoint. It runs only on Rockchip RK3576 with two cores.

License and attribution

This repository contains a quantized RKLLM conversion of IBM Granite 4.2 3B:

https://huggingface.co/ibm-granite/granite-4.2-3b

The original model and tokenizer are provided by IBM under the Apache License 2.0.

The tokenizer files in this repository are copied from the original IBM Granite 4.2 3B repository without modification.

This repository contains an unofficial converted and quantized derivative. It is not an official IBM release.