pocketllm-models

PocketLLM Model Collection

<div align="center">

The official model collection for PocketLLM

Private • Offline • On-Device AI for Android

</div>

---

What is PocketLLM?

PocketLLM is an Android app that runs large language models completely on your device — no internet required during inference, no data sent to the cloud, no subscriptions.

This repository hosts the curated model collection optimized for mobile edge inference using:

• MediaPipe LLM Inference (for .bin and .task files — Gemma family)
• llama.cpp via llama.rn (for .gguf files — Llama, Phi, Qwen, SmolLM, Gemma 3)

---

Model Catalog

⚡ Ultra Fast — Under 1 GB, fits any Android phone

⚖️ Balanced — 1–2 GB, great quality on mid-range phones

🚀 Powerful — Best quality, needs 5GB+ RAM

---

How to Use in PocketLLM

Models are downloaded directly inside the PocketLLM app. Open the Model Store tab, select a model, and tap Download. The app handles everything automatically.

Direct download URLs:

https://huggingface.co/AmareshHebbar/pocketllm-models/resolve/main/<filename>

---

Fine-Tuning Roadmap

Coming soon — within the next 2 weeks

We are fine-tuning these base models specifically for on-device conversational AI on mobile:

Goals

• Better persona adherence (the model stays in character consistently)
• Shorter, more natural responses (base models tend to be verbose on mobile)
• Improved memory utilization (uses injected memories naturally)
• Better instruction following for small context windows (2048 tokens)
• Hindi + English code-switching support (for India market)

Training approach

• Base: Llama 3.2 1B, Gemma 3 1B (smallest models first — fastest iteration)
• Method: QLoRA fine-tuning (4-bit, similar to the AxioMapper training setup)
• Dataset: Curated conversational dataset optimized for mobile edge constraints
• Hardware: Single A100 via RunPod
• Framework: Unsloth (2x faster fine-tuning, same as AxioMapper)

Fine-tuned models (coming soon)

• pocketllm-llama-1b-v1.gguf — Llama 3.2 1B fine-tuned for mobile chat
• pocketllm-gemma-1b-v1.gguf — Gemma 3 1B fine-tuned for persona consistency

---

Model Selection Guide

Your phone has...          Best model to start with
─────────────────────────────────────────────────────
2 GB RAM (budget)    →    Qwen 2.5 0.5B  (fastest)
3 GB RAM             →    Llama 3.2 1B   (balanced speed)
4 GB RAM (mid-range) →    Gemma 1.1 2B   (best all-rounder)
6 GB RAM             →    Llama 3.2 3B   (better quality)
8 GB RAM (flagship)  →    Phi-3.5 Mini   (best for coding)

---

Technical Details

MediaPipe format (.bin, .task)

• Used for Gemma family
• Runs via Google's MediaPipe LLM Inference SDK
• Supports CPU and GPU acceleration (Vulkan/OpenCL)
• Integrated via react-native-llm-mediapipe

GGUF format (.gguf)

• Used for Llama, Qwen, Phi, SmolLM, Gemma 3
• Runs via llama.cpp (the gold standard for mobile inference)
• Q4_K_M quantization — best balance of size and quality
• Integrated via llama.rn

---

Repository Structure

pocketllm-models/
├── gemma-1.1-2b-it-cpu-int4.bin       ← Gemma 2B CPU (MediaPipe)
├── gemma-1.1-2b-it-gpu-int4.bin       ← Gemma 2B GPU (MediaPipe)
├── gemma-3-1b-it-q4_k_m.gguf         ← Gemma 3 1B (GGUF)
├── gemma-3-4b-it-q4_k_m.gguf         ← Gemma 3 4B (GGUF)
├── llama-3.2-1b-instruct-q4_k_m.gguf ← Llama 3.2 1B (GGUF)
├── llama-3.2-3b-instruct-q4_k_m.gguf ← Llama 3.2 3B (GGUF)
├── phi-3.5-mini-instruct-q4_k_m.gguf ← Phi-3.5 Mini (GGUF)
├── qwen2.5-0.5b-instruct-q4_k_m.gguf ← Qwen 2.5 0.5B (GGUF)
└── smollm2-1.7b-instruct-q4_k_m.gguf ← SmolLM2 1.7B (GGUF)

---

License

• Gemma models: Gemma Terms of Use
• Llama models: Meta Llama 3.2 Community License
• Phi models: MIT License
• Qwen models: Apache 2.0
• SmolLM2: Apache 2.0
• Fine-tuned models by PocketLLM: Apache 2.0

---

Built By

Amaresh Hebbar

Building PocketLLM: the only mobile app that runs a full AI agent stack — smart routing, persona memory, MCP tools — completely offline on Android.

"Your AI. Your phone. Nobody else's business."