Holo-3.1-4B-GGUF

Holo-3.1-4B-GGUF

Holo3.1: Fast & Local Computer Use Agents

Model Description

Holo3.1 is our latest family of Vision-Language Models (VLMs) for computer use agents. Building on Holo3, it expands support beyond browser and desktop automation to mobile environments, introduces native function-calling support for seamless integration with agent frameworks, and enables local deployment through optimized quantized checkpoints.

The Holo3.1 family spans model sizes from 0.8B to 35B-A3B parameters. Across computer use, UI grounding, mobile automation, and business workflows, Holo3.1 delivers strong performance while improving deployment flexibility and cost efficiency.

[!NOTE] For more information, please visit the original model card: https://huggingface.co/Hcompany/Holo-3.1-4B

---

Model Files

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

FROM ghcr.io/ggml-org/llama.cpp:full

WORKDIR /app

RUN apt update && apt install -y python3-pip
RUN pip install -U huggingface_hub --break-system-packages

RUN python3 -c 'from huggingface_hub import hf_hub_download; \
    repo="prithivMLmods/Holo-3.1-4B-GGUF"; \
    hf_hub_download(repo_id=repo, filename="Holo-3.1-4B.Q5_K_M.gguf", local_dir="/app"); \
    hf_hub_download(repo_id=repo, filename="Holo-3.1-4B.mmproj-bf16.gguf", local_dir="/app")'

CMD ["--server", \
     "-m", "/app/Holo-3.1-4B.Q5_K_M.gguf", \
     "--mmproj", "/app/Holo-3.1-4B.mmproj-bf16.gguf", \
     "--host", "0.0.0.0", \
     "--port", "7860", \
     "-t", "2", \
     "--cache-type-k", "q8_0", \
     "--cache-type-v", "iq4_nl", \
     "-c", "128000", \
     "-n", "38912"]