SmolLM2-1.7B-Instruct

SmolLM2-1.7B-Instruct (LiteRT-LM)

This repository hosts the LiteRT-LM (LiteRT, formerly known as TensorFlow Lite) version of HuggingFaceTB/SmolLM2-1.7B-Instruct, optimized for on-device inference across Android, desktop, and edge hardware.

Available Models

• SmolLM2-1_7B-Instruct_dynamic_wi8_afp32.litertlm:
  • Quantization: Dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32).
  • Hardware Optimization: Converted via litert-torch with boolean attention masks and fused GPU composite operations enabled for high-throughput GPU and CPU execution.

---

Use the Model

1. Edge Gallery App (Android)

You can run this model directly on Android using the official Google AI Edge Gallery app:

1. Get the App: Install the app from Google Play or download the latest APK from the GitHub releases page.
2. Import the Model: Navigate to the Model manager within the app and tap the "+" (plus) icon in the bottom-right corner:
   • Import from HF (Recommended): Select this option, enter litert-community/SmolLM2-1.7B-Instruct, and submit to download directly inside the app.
   • From local model file: Download SmolLM2-1_7B-Instruct_dynamic_wi8_afp32.litertlm directly to your Android device (or push it via ADB: adb push SmolLM2-1_7B-Instruct_dynamic_wi8_afp32.litertlm /sdcard/Download/), select the file from storage, choose your preferred backend (GPU or CPU), and tap "Import". For full details on importing models and building from source, see the Edge Gallery App Wiki.

2. Try It (Desktop / CLI)

Install uv and run the model directly from the LiteRT-LM command line:

uv tool install litert-lm
uvx litert-lm run \
  --from-huggingface-repo=litert-community/SmolLM2-1.7B-Instruct \
  SmolLM2-1_7B-Instruct_dynamic_wi8_afp32.litertlm \
  --prompt="What is the capital of France?"