Potato-CODER-12GB-Fast

Potato-CODER-12GB-Fast (v1.0-rc1)

The fast 12 GB Potato: a 35B-A3B model whose weights are about 90% ternary. The model, 131,072-token KV cache and vision projector run on the GPU with no CPU weight offloading. On an RTX 3090 held to an 11,836 MiB VRAM envelope, it decodes at about 170 tok/s and prefills at about 2,200 tok/s. Six fixed tasks finished 2.4× faster than with Potato-CODER-12GB.

The language-model weights are LLMs For All's Millie-1.1-35B-A3B-Ternary, unmodified. What is ours: the CUDA kernels that make it run at this speed on NVIDIA hardware, a Q8_0 vision projector converted from theirs, the terse chat template that shortens its answers, and the deployment recipe.

This is the faster choice for interactive work; the standard version is stronger on multi-step repository work.

Files

Run on Linux (NVIDIA)

Use the Fast CUDA runtime on Linux x64 (glibc 2.35+) with an NVIDIA 570+ driver. Install the Hugging Face CLI first. The Hub's autogenerated :Q8_0 command and the regular Potato binaries cannot serve this ternary model; Q8_0 is the projector.

hf download Slomin/Potato-CODER-12GB-Fast --revision fast-v1.0.0-rc1 --local-dir potato-fast
cd potato-fast
runtime=https://github.com/potato-os/llama.cpp/releases/download/potato-millie-v1.0.0-rc3
name=llama-potato-millie-linux-cuda-12.8-x64
for file in "$name.tar.gz" "cudart-$name.tar.gz" SHA256SUMS.txt; do curl -fLO "$runtime/$file"; done
grep "$name" SHA256SUMS.txt | sha256sum -c -
tar -xzf "$name.tar.gz" --strip-components=1
tar -xzf "cudart-$name.tar.gz" --strip-components=1
sha256sum -c SHA256SUMS
bash serve-potato.sh

In another terminal, check curl -fsS http://127.0.0.1:8080/health. Point an OpenAI-compatible client at http://127.0.0.1:8080/v1 with model ID potato-coder-12gb-fast. The script sets one slot and 131,072 tokens.

Run on Windows (NVIDIA)

Use the Fast CUDA 12.4 release on Windows x64 with an NVIDIA 551+ driver. Install the Hugging Face CLI, then run in PowerShell:

$ErrorActionPreference='Stop'
hf download Slomin/Potato-CODER-12GB-Fast --revision fast-v1.0.0-rc1 --local-dir potato-fast
Set-Location potato-fast
$runtime='https://github.com/potato-os/llama.cpp/releases/download/potato-millie-v1.0.0-rc3'
$name='llama-potato-millie-win-cuda-12.4-x64'
foreach ($file in @("$name.zip", "cudart-$name.zip", 'SHA256SUMS.txt')) { Invoke-WebRequest "$runtime/$file" -OutFile $file }
$checks=Get-Content SHA256SUMS.txt
foreach ($file in @("$name.zip", "cudart-$name.zip")) { if ("$((Get-FileHash $file).Hash.ToLower())  $file" -notin $checks) { throw "Checksum failed: $file" } }
Expand-Archive "$name.zip" -DestinationPath . -Force
Expand-Archive "cudart-$name.zip" -DestinationPath . -Force
foreach ($line in Get-Content SHA256SUMS) { $sum,$file=$line -split '  ',2; if ((Get-FileHash $file).Hash.ToLower() -ne $sum) { throw "Checksum failed: $file" } }
.\serve-potato.bat

Check http://127.0.0.1:8080/health. Use http://127.0.0.1:8080/v1 and model ID potato-coder-12gb-fast in your client. Linux RTX 3090 GPU inference is tested; Windows CI builds the CUDA backend and checks CPU inference and the launcher without a GPU.

The knobs that matter

• The terse template. chat-template-sharp-potato.jinja is peculiar-ragdoll's Sharp template (Apache-2.0), which force-appends a brevity system block. This model is verbose by default — it tends to reach its answer early and then re-check it several times over — and the block shortens that without changing how it answers. Nothing was trained to get it; it is a prompt. (The only change we made is the two sentences that name the model, so it does not introduce itself as somebody else's model.)
• --presence-penalty 0, not 1.5. The upstream inference-settings.json recommends 1.5 for thinking, and for chat or completion use that is a reasonable setting. For agent use it is not: an agent repeats file paths and re-issues similar tool calls on purpose, and penalising that costs solved tasks.
• Thinking budget 16,384 + the wrap-up message. The budget alone truncates a long chain; the message is what converts it into a finished answer.
• 131,072 is the top rung, and the projector is why. The published projector has no GPU kernels; our F16 conversion of it runs, but at 861 MiB it left room for only 114,688 tokens. At Q8_0 it is 586 MiB, and the 275 MiB that buys is worth exactly one rung: 131,072 passes every gate with 462 MiB spare. 147,456 also passes, with 258 MiB — eight over our own ≥250 MiB line, which is not a margin to ship. 163,840 fails on that line alone. 462 MiB is the real margin here: the card must be otherwise empty.
• One slot. Every number here is single-slot.

How it was put together

19–20 September 2026, on RTX 3090s held down to a 12 GB envelope. Six candidate models were served in that envelope and compared on fit, context depth, speed and agentic coding; this one was kept as the fast half of the pair. The configuration above is the highest context rung that passed every fit gate: single-slot property checks, retrieval at half and full depth, a near-limit turn, three repeated image rounds (screenshots and photographs, including after a deep context fill), a VRAM-growth check and a projector-placement check.

Our comparison numbers are internal and not published here.

Limits

• It is the weaker of the two Potatoes on multi-step repository work. That is the trade this model exists to make.
• The CUDA kernels are ours. They were validated against the model's CPU reference (their divergence is no larger than the spread two numerically different backends of the same build show), plus dequantization edge tests and a bounded sanitizer run, and they have been exercised on sm_86 (RTX 3090) only.
• The Q8_0 projector is a conversion of the published one, not a re-training. Its quantization error is real (max about 0.002 absolute, about 0.6% RMS relative on the quantized tensors), and what we checked is that vision still works: screenshots and photographs read correctly in every fit gate, including after a full-depth context fill, at the same encode time as the F16 file. We have not measured perceptual image quality against the F16 or the original projector.
• The 462 MiB of spare VRAM is measured on a headless RTX 3090 held to a 12 GB envelope; other cards, drivers and desktops differ by a few hundred MiB.
• Everything was measured on Linux.

Licence and attribution

Apache-2.0 (LICENSE, NOTICE).

• Qwen — Qwen3.6-35B-A3B, the base model (Apache-2.0).
• LLMs For All, Inc. — Millie 1.1: the ternary compression, the weights and the Millie Runtime (Apache-2.0 weights, MIT runtime).
• peculiar-ragdoll — the Sharp chat template (Apache-2.0), shipped here with the two model-name sentences changed.
• Slomin — CUDA kernels for the ternary tensor type, the Q8_0 projector conversion, the 12 GB deployment recipe and the fit testing behind it. No language-model weights were trained or requantized.