Downloads · 30 days
12
3% of all-time downloads
nightmedia/granite-4.1-8B-Holodeck-q8-hi-mlx
granite-4.1-8B-Holodeck-q8-hi-mlx is a text generation model from nightmedia. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
This model is a NuSLERP merge of: - DavidAU/granite-4.1-8b-Brainstone-Thinking - nightmedia/granite-4.1-8B-TNG-Coder
Downloads · 30 days
12
3% of all-time downloads
All-time downloads
398
Public
Parameters
8.8B
9.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors9.9 GB · 100%
How the weights are stored.
U328.8B · 100%
From the Hugging Face model README
This model is a NuSLERP merge of:
quant arc arc/e boolq hswag obkqa piqa wino
q8-hi 0.557,0.723,0.874
Quant Perplexity Peak Memory Tokens/sec
q8-hi 4.083 ± 0.028 12.99 GB 652
granite-4.1-8b-Brainstone-Thinking
quant arc arc/e boolq hswag obkqa piqa wino
mxfp8 0.531,0.701,0.863
granite-4.1-8B-TNG-Coder
quant arc arc/e boolq hswag obkqa piqa wino
mxfp8 0.538,0.723,0.853,0.701,0.478,0.782,0.703
qx86-hi 0.536,0.713,0.861
qx64-hi 0.532,0.715,0.861,0.701,0.454,0.777,0.693
q8-hi 0.531,0.712,0.859,0.710,0.464,0.782,0.695
q6-hi 0.537,0.711,0.859,0.710,0.464,0.776,0.689
q4-hi 0.532,0.705,0.863,0.698,0.456,0.774,0.682
Quant Perplexity Peak Memory Tokens/sec
qx86-hi 4.070 ± 0.027 11.83 GB 605
qx64-hi 4.141 ± 0.027 9.63 GB 651
q8-hi 4.098 ± 0.027 12.99 GB 574
q6-hi 4.100 ± 0.027 10.80 GB 678
q4-hi 4.215 ± 0.027 8.60 GB 712
mxfp8 4.460 ± 0.029 12.17 GB 642
mxfp4 4.819 ± 0.033 7.77 GB 714
models:
- model: DavidAU/granite-4.1-8b-Brainstone-Thinking
parameters:
weight: 1.6
- model: nightmedia/granite-4.1-8B-TNG-Coder
parameters:
weight: 0.4
merge_method: nuslerp
dtype: bfloat16
name: granite-4.1-8B-Brainstone-TNG-Coder2
name: granite-4.1-8B-Holodeck
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("granite-4.1-8B-Holodeck-q8-hi-mlx")
prompt = "hello"
if tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_dict=False,
)
response = generate(model, tokenizer, prompt=prompt, verbose=True)