Downloads · 30 days
0
Mixtral-8x7B-v0.1-hf-4bit-mlx-adapters
mzbac/Mixtral-8x7B-v0.1-hf-4bit-mlx-adapters
Mixtral-8x7B-v0.1-hf-4bit-mlx-adapters is a machine learning model from mzbac. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Access
Public
Updated Jan 26, 2024
Repo size
71.4 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.npz35.7 MB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Jan 26, 2024
- Updated
- Jan 26, 2024
- SHA
- d53eb333
Model card
From the Hugging Face model README
Qlora adapters for Mixtral-8x7B-v0.1-hf-4bit-mlx
fine-tuned on guanaco dataset
inference vis mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Mixtral-8x7B-v0.1-hf-4bit-mlx",adapter_file="adapters.npz")
generate(model=model, tokenizer=tokenizer, prompt="### Human: write a quick sort in python.\n### Assistant: ", max_tokens=500, verbose=True,temp=0.3)
serve as an API Service
pip install mlx-llm-server
mlx-llm-server --model-path mlx-community/Mixtral-8x7B-v0.1-hf-4bit-mlx --adapter-file adapters.npz
- License
- mit
- Created
- Jan 26, 2024
- Updated
- Jan 26, 2024