DeepSeek-V3.2-RAM-350GB-MLX

DeepSeek-V3.2 — 350 GB (MLX)

Mixed-precision MLX build of deepseek-ai/DeepSeek-V3.2, prepared by baa.ai.

Built at the predicted quality-maximum operating point — the budget at which additional bits stop delivering measurable quality gains.

Metrics

Benchmarks

Benchmarks are pending — this model requires a 384 GB+ Mac to load.

Recommended inference settings

sampler_params = {
    "temperature": 1.0,
    "top_p": 0.95,
    "top_k": 40,
    "repetition_penalty": 1.1,
    "max_tokens": 8192,
}

Chat template — thinking mode

DeepSeek-V3.2 uses a <think>...</think> reasoning block. The model begins its response inside a thinking block. Strip everything up to and including the first </think> to get the final answer:

def strip_thinking(text: str) -> str:
    if "</think>" in text:
        return text.split("</think>", 1)[1].strip()
    return text.strip()

Note: DeepSeek-V3.2 does not ship a Jinja chat template. For advanced chat formatting (tool calling, multi-turn), refer to the encoding scripts in the source repo.

Usage

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler, make_logits_processors

model, tokenizer = load("baa-ai/DeepSeek-V3.2-RAM-350GB-MLX")

sampler = make_sampler(temp=1.0, top_p=0.95, top_k=40)
logits_processors = make_logits_processors(repetition_penalty=1.1)

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Write a Python function that reverses a string."}],
    tokenize=False,
    add_generation_prompt=True,
)

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=8192,
    sampler=sampler,
    logits_processors=logits_processors,
)

if "</think>" in response:
    response = response.split("</think>", 1)[1].strip()
print(response)

Hardware

• Apple Silicon Mac with 384 GB+ unified memory required.
• Fits comfortably on a 512 GB Mac Studio Ultra or Mac Pro.
• Does not fit on 192 GB machines without heavy swap.

Variants

Black Sheep AI Products

Black Sheep AI builds private AI with the Bounded AI Architecture (BAA): three layers, each bound to something you control, on infrastructure you control. Nothing leaves your boundary.

Ram (Reasoning Layer) — Full-capability frontier models, shrunk to fit smaller GPUs with reasoning kept and certified. They rely strictly on your Data Modules for knowledge, so you swap the model as better ones ship.

Paddock (Knowledge Plane) — Binds your Data Modules directly to the reasoning layer: one per topic, period, person, permission or tenant. Permission, freeze, hand back or delete each one on its own.

Watchman (GRC Layer) — Governance, risk and compliance in a single, complete audit log: every module permission, every bind, every model's provenance. Five certificates before anything goes live.

Shepherd — Takes a frontier model, compresses it into a Ram reasoning model, certifies it with Watchman, binds it to your Paddock, and deploys the Bounded AI Architecture to your infrastructure: your cloud, a Mac Studio, or an air-gapped facility.

Learn more at baa.ai — Private AI for enterprises whose data cannot leave the building.

License

MIT — inherited from deepseek-ai/DeepSeek-V3.2.

---

Quantized by baa.ai