JOSIE-1.1-4B-Instruct-litert-lm

JOSIE-1.1-4B-Instruct_DYNAMIC WI4 AFP32 — LiteRT-LM

On-device .litertlm conversion of Goekdeniz-Guelmez/JOSIE-1.1-4B-Instruct for the LiteRT-LM runtime. Runs completely offline on Android, Pixel, and Google Tensor / Mali-based devices.

---

What This Model Is

This is a community-packaged variant of the model provided for on-device inference via the LiteRT-LM engine.

Key characteristics:

• Text-only — image and audio understanding weights were removed to keep the bundle small and optimized for text tasks.
• 4096 context window — configured for local inference memory efficiency.
• WI4 quantized — optimized for mobile storage with balanced precision.
• On-device — runs completely offline on supported Android / Pixel / Tensor devices.

How to Use

Google AI Edge Gallery (Pixel / Android)

1. Install Google AI Edge Gallery or a compatible fork.
2. Add model via HuggingFace URL: Werve/JOSIE-1.1-4B-Instruct-litert-lm
3. Download the 2.09 GB bundle over WiFi.
4. Chat offline — no data leaves your device.

LiteRT-LM CLI (macOS / Linux / Desktop)

pip install litert-lm
litert-lm run --from-huggingface-repo Werve/JOSIE-1.1-4B-Instruct-litert-lm \
              JOSIE-1.1-4B-Instruct_DYNAMIC WI4 AFP32.litertlm \
              --prompt "Your prompt here" \
              --backend gpu

---

Technical Details

Export Parameters

• Quantization: dynamic_wi4_afp32
• Context Length: 4096
• Export Library: litert-torch 0.9.1+

Weight Extraction

Only text-decoder safetensors were retained. Multimodal components (vision/audio) were stripped to reduce total size from the original checkpoint.

---

Credits & Attribution

• Original Model: Goekdeniz-Guelmez/JOSIE-1.1-4B-Instruct
• LiteRT-LM packaging: Werve

This is an unofficial community build. It is not published by Google or the official LiteRT team.