GPT-OSS-Swallow-120B-RL-v0.1-6bit-mlx

GPT-OSS-Swallow-120B-RL-v0.1 — MLX 6-bit

6-bit quantized MLX version of tokyotech-llm/GPT-OSS-Swallow-120B-RL-v0.1, optimized for Apple Silicon.

Key Details

• Architecture: gpt_oss (Mixture of Experts — 117B total, 5.1B active)
• Quantization: 6-bit (6.504 bits/weight)
• Disk size: ~89 GB
• Peak memory: ~95 GB
• Generation speed: ~74 tok/s on M4 Max 128GB
• Requires: 128GB unified memory (Apple Silicon)

Usage

pip install mlx-lm

mlx_lm.generate \
  --model tocchitocchi/GPT-OSS-Swallow-120B-RL-v0.1-6bit-mlx \
  --prompt "日本の首都はどこですか？" \
  --max-tokens 200

from mlx_lm import load, generate

model, tokenizer = load("tocchitocchi/GPT-OSS-Swallow-120B-RL-v0.1-6bit-mlx")

messages = [{"role": "user", "content": "Pythonでフィボナッチ数列を出力するコードを書いてください"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)

response = generate(model, tokenizer, prompt=prompt, verbose=True, max_tokens=500)

License

Apache 2.0 (inherited from base model)