Millie-1.1-35B-A3B-Ternary

Millie-1.1-35B-A3B-Ternary

Millie 1.1 is a language and vision model from LLMs For All, built for local inference on memory-constrained devices. It has approximately 35 billion total parameters and 3 billion active parameters per token. Roughly 90% of its weights use a ternary representation; the remaining tensors use higher precision.

These are the weights used by the flagship model in the initial release of the Millie iPhone app.

Files

Sizes are decimal GB. Disk size is not runtime RAM usage. The vision file is optional for text-only use.

Run Millie

1. Download the archive for your platform from Millie Runtime v0.1.0 and extract it.
2. Download Millie-1.1-35B-A3B-Ternary.gguf from this repository.
3. Open a terminal in the extracted runtime folder and run:

./bin/llama-cli \
  -m /path/to/Millie-1.1-35B-A3B-Ternary.gguf \
  --jinja --reasoning off \
  --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0 \
  --presence-penalty 1.5 --repeat-penalty 1

Replace /path/to/ with the folder where you downloaded the model, then type a message to chat.

For images, also download Millie-1.1-35B-A3B-vision.gguf and add:

--mmproj /path/to/Millie-1.1-35B-A3B-vision.gguf --image /path/to/photo.jpg

The runtime supports ordinary inference and a low-memory mode that keeps a bounded number of experts resident per layer. Low-memory mode currently requires Apple Metal. See low-memory setup.

Generation settings

Millie inherits Qwen's sampling defaults for non-thinking and thinking generation:

Both use min-p 0, presence penalty 1.5, and repetition penalty 1.0. For thinking, use --reasoning on --temp 1.0 --top-p 0.95 instead.

Evaluation

We compare the Qwen 3.6 teacher, the compressed model with unrestricted expert routing, and the compressed model in a low-memory configuration.

The low-memory configuration uses 64 hot experts per layer, a +1 routing-logit bias for hot experts, and a simulated 8-bit (Q8_0) KV cache. The evaluations used single sampled runs with up to 64K generated tokens per example.

Attribution and license

Millie is based on Qwen3.6-35B-A3B. The model files declare Apache 2.0, matching the upstream license; a copy is included in this package. LLMs For All modified and compressed the model and provides the corresponding runtime support. The runtime is distributed separately under its MIT license.