sally-1.0-GGUF

Sally v1.0 GGUF

Quantized build of sallya1c/sally-1.0, ready to run locally. 7.33 GB, one file, no GPU required.

Sally is a specialized intelligence for metabolic health. Sally answers with a position rather than a summary: which magnesium to take, whether meal order matters, what to do about a glucose pattern.

This repository is the reasoning layer only. It has no biomarker ingestion, no source citations and no skills; those belong to the Sally AI platform. Running this locally does not reproduce the product.

Quick start with Ollama

hf download sallya1c/sally-1.0-GGUF sally-1.0-Q3_K_M.gguf Modelfile --local-dir sally
cd sally
ollama create sally -f Modelfile
ollama run sally "Is sugar healthy?"

Three things to know about those commands:

• hf comes from pip install huggingface_hub. No login is needed; this repository is public.
• Download both files. The Modelfile carries the system prompt and sampling defaults.
• Run ollama create from the folder containing the .gguf, because the Modelfile refers to it by relative path. That is what the cd sally is for.

Once created, the model is available anywhere Ollama is: the CLI, the desktop app's model picker, and the HTTP API on localhost:11434.

curl http://localhost:11434/api/chat -d '{
  "model": "sally",
  "messages": [{"role": "user", "content": "Which form of magnesium should I take?"}],
  "stream": false
}'

Alternative: llama.cpp

hf download sallya1c/sally-1.0-GGUF sally-1.0-Q3_K_M.gguf --local-dir .

llama-cli -m sally-1.0-Q3_K_M.gguf -ngl 99 -c 8192 --jinja \
  -sys "You are Sally, an AI metabolic strategist. You prioritise glucose stability and insulin sensitivity over convenience or popularity. State a clear position and a concrete alternative rather than listing pros and cons. Never advise on medication dosing; refer those to a clinician."

The system prompt is not optional

Served with no system prompt, this model answers like the base Qwen3-14B it was tuned from: it recommends oat milk, frames weight loss as a calorie deficit, and will discuss insulin-to-carb ratios. The prompt in the Modelfile is what produces Sally's behaviour.

That prompt contains no proprietary protocol content, so replace it with your own framing if you prefer. Just do not omit it.

On a 10-case protocol check, the prompted model scored 9 of 10 aligned with zero contradictions, against 6 of 10 with two contradictions for the unmodified base model.

What to expect from your hardware

Generation is memory-bandwidth bound: every token requires reading all 7.33 GB of weights, so the ceiling is roughly bandwidth / 7.33 GB, and real throughput lands near 60 to 65% of that.

Only the first row is measured; the rest scale from bandwidth and are estimates. Time to first token is well under a second on a normal question, and a warm model loads in under 0.1s.

Memory needed: 7.33 GB of weights plus the KV cache. The KV cache costs 160 KiB per token (40 layers, 8 KV heads, head_dim 128), so 8k context adds about 1.25 GB, for roughly 9 GB total. A 16 GB machine is comfortable. On 12 GB, lower num_ctx to 4096 or set OLLAMA_KV_CACHE_TYPE=q8_0, which halves the cache.

Do not raise num_ctx to the model's 40,960 maximum: the full window alone is 6.4 GB of KV.

Files

Q3_K_M was chosen to stay under 8 GB. Importance-matrix calibration was used, which llama.cpp recommends for anything below Q6. Higher-precision builds of the same merge exist and behave slightly better; open an issue if you want one published.

Reproducing the build

Merge the root adapter only from the source repository. The two adapters published there are 0.9997 cosine-similar per tensor, because stage 2 was initialised from stage 1 and therefore already contains it. Applying both stacks the same update twice and destroys output quality at every quantization level.

llama-export-lora -m Qwen3-14B-BF16.gguf --lora sally-root-lora.gguf -o merged.gguf
llama-quantize --imatrix imatrix.gguf merged.gguf sally-1.0-Q3_K_M.gguf Q3_K_M

Note that llama-export-lora is not included in every llama.cpp distribution and may need building from source.

Notes and limitations

• Not medical advice. All medication, diagnostic and treatment decisions belong with a licensed clinician. Safety boundaries are trained in, not guaranteed; enforce them in your application layer.
• Prompt-dependent safety. With no system prompt, the type 1 diabetes refusal does not hold. Use the provided Modelfile or an equivalent.
• No data access, no retrieval. These weights cite no sources and connect to no biomarker feed.
• Quantization trade-off. Q3_K_M gives up some quality for size and speed.
• Reasoning tokens. The base architecture emits <think> blocks by default. Append /no_think to a message, or pass "think": false through the API.

Built by a1c.io. Platform: console.a1c.io.