Downloads · 30 days
117
7% of all-time downloads
lokinfey/gpt-oss-20B-mlx-metal-int4
gpt-oss-20B-mlx-metal-int4 is a text generation model from lokinfey. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
This is a quantized INT4 model based on Apple MLX Framework gpt-oss-20B. You can deploy it on Apple Silicon devices (M1,M2,M3,M4).
Downloads · 30 days
117
7% of all-time downloads
All-time downloads
1.8K
Public
Parameters
20.9B
11.8 GB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors11.8 GB · 100%
How the weights are stored.
U3220.9B · 100%
From the Hugging Face model README
This is a quantized INT4 model based on Apple MLX Framework gpt-oss-20B. You can deploy it on Apple Silicon devices (M1,M2,M3,M4).
<b><span style="text-decoration:underline">Note: This is unoffical version,just for test and dev.</span></b>
pip install -U mlx-lm
python -m mlx_lm.convert --hf-path openai/gpt-oss-20b -q
from mlx_lm import load, generate
model, tokenizer = load("Your gpt-oss-20B-mlx-metal-int4 Path")
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "can you introduce yourself"}],
tokenize=False,
add_generation_prompt=True,
)
response = generate(model, tokenizer, prompt=prompt,max_tokens=1024, verbose=True)