garden

[!TIP] KV-cache quantization needs no fork (2026): upstream llama.cpp / Ollama cover it natively. Use -ctk q8_0 -ctv q8_0 (~half KV memory, perplexity +0.002–0.05) or -ctk q4_0 -ctv q4_0 (~quarter memory, ≈7.6% perplexity increase). In Ollama: OLLAMA_KV_CACHE_TYPE=q8_0 with OLLAMA_FLASH_ATTENTION=1. Keep K and V types symmetric to stay on the fused Flash-Attention path. Since April 2026 mainline llama.cpp also applies Hadamard rotation to KV activations (PR #21038), which greatly improves low-bit KV quality (opt-out: LLAMA_ATTN_ROT_DISABLE=1).

<!-- kv-upstream-note -->

majentik — Model Garden

Quantized open-weight models for Apple Silicon and llama.cpp, released only after a coherence smoke gate passes on real hardware. Every repo keeps the upstream tokenizer, architecture and license; the only thing we change is how the weights are stored.

402 model repositories · 14 datasets · 13 collections · snapshot 2026-09-16

How releases are made

1. Pin upstream. Every lane pins the exact upstream revision it quantized from and refuses to run if the source drifts.
2. Quantize with standard tooling. mlx-lm / mlx-vlm / mlx-audio for MLX, llama.cpp for GGUF. Group size 32 for ≤4-bit MLX tiers. We never publish a tier at or above the source's bits-per-weight.
3. Smoke gate. Each tier must produce coherent greedy output on a fixed prompt set before upload. Tiers that fail are held back and listed as such on the family card instead of being shipped — expect gaps in the tier ladder for small or fragile models.
4. Evidence. Gate records are published in majentik/garden-quant-bench. Repos in the Verified MLX releases collection link their evidence directly.

Lanes

Repo naming is unbranded: majentik/<Model>-MLX-<tier> and majentik/<Model>-GGUF-<QT>.

Families

Counts are repos on the Hub as of the snapshot date.

Datasets

Calibration and evaluation sets used by the lanes, mirrored so results are reproducible: ultrachat-calib, c4-calib, ultra-fineweb-calib, ultradata-{sft,math}-calib, tulu-3-sft-mixture, wikitext-2-ppl, gsm8k, ifeval, fleurs-ar-asr, WildASR, SASRBench-v1, magpie-reasoning-qwen25-7b, and the gate ledger garden-quant-bench.

Pick a starting point

• MacBook, 16 GB → Gemma 4 E4B-it MLX 4-bit, or LFM2.5-8B-A1B MLX 4-bit.
• MacBook, 32–64 GB → Qwen3.6-35B-A3B MLX 4-bit, Nemotron-3-Nano-30B-A3B MLX 4-bit.
• Mac Studio, 128 GB+ → Qwen3.5-122B-A10B MLX 4-bit, Mistral-Medium-3.5-128B MLX 4-bit, gpt-oss-120b.
• Speech on Apple Silicon → MERaLiON-3-3B-ASR MLX 8-bit (Singapore English / Malay / Chinese / Tamil), Qwen3-ASR MLX.
• Long context on llama.cpp → any GGUF tier plus -ctk q8_0 -ctv q8_0 (see tip above).

What is not in each repo

• Training data. Quantization-only releases; the base model's training data and disclaimers are upstream's.
• Benchmarks for every axis. We publish the smoke gate result, and where measured, WikiText-2 perplexity or family-specific evals (KL vs. bf16, WER for ASR). Missing an axis you need? Open a discussion on the repo.

About the RotorQuant / TurboQuant labels

Older repos in this org carry RotorQuant or TurboQuant in their names. These are historical release labels, not distinct quantization algorithms: for any given tier both labelled repos hold byte-identical weights produced with the standard MLX / llama.cpp quantizers, and no brand-specific speedup is claimed or measured. New releases are unbranded. The associated llama.cpp fork is unmaintained; use upstream KV-cache quantization as described in the tip at the top.

Who we are

majentik publishes these to keep our own fleet running cheaply on commodity Apple hardware and to close the gap between a research release and "can I actually run this tonight". Issues, quant requests and benchmark PRs are welcome via the Community tab on the closest repo.

Versioning

Each repo tracks upstream@base-model-revision × quant-lane. When upstream ships a new base revision we re-run the lane and bump the repo. Card-only changes do not bump the version.

License

Each repo inherits the base model's license, not this organization-level license. Check the license field in the repository's card before deploying.