Downloads · 30 days
24
9% of all-time downloads
Llama-3.2-1B-Instruct-gptqmodel-4bit-vortex-mlx-v2.5
ModelCloud/Llama-3.2-1B-Instruct-gptqmodel-4bit-vortex-mlx-v2.5
Llama-3.2-1B-Instruct-gptqmodel-4bit-vortex-mlx-v2.5 is a machine learning model from ModelCloud. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This model was quantized and exported to mlx using GPTQModel.
All-time downloads
271
Public
Parameters
232M
790 MB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors773 MB · 98%
Parameter types
How the weights are stored.
U32154M · 67%
Model card
From the Hugging Face model README
This model was quantized and exported to mlx using GPTQModel.
How to run this model
# install mlx
pip install mlx_lm
from mlx_lm import load, generate
mlx_path = "ModelCloud/Llama-3.2-1B-Instruct-gptqmodel-4bit-vortex-mlx-v2.5"
mlx_model, tokenizer = load(mlx_path)
prompt = "The capital of France is"
)
text = generate(mlx_model, tokenizer, prompt=prompt, verbose=True)
Export gptq to mlx
# install gptqmodel with mlx
pip install gptqmodel[mlx] --no-build-isolation
from gptqmodel import GPTQModel
# load gptq quantized model
gptq_model_path = "ModelCloud/Llama-3.2-1B-Instruct-gptqmodel-4bit-vortex-v2.5"
mlx_path = f"./vortex/Llama-3.2-1B-Instruct-gptqmodel-4bit-vortex-mlx-v2.5"
# export to mlx model
GPTQModel.export(gptq_model_path, mlx_path, "mlx")
- Type
- llama
- License
- apache-2.0
- Created
- Jan 16, 2025
- Updated
- Jan 24, 2025