Downloads · 30 days
28
13% of all-time downloads
allura-quants/Luna-27B-v0_mlx-hi
Luna-27B-v0_mlx-hi is a image-text-to-text model from allura-quants. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as gemma.
MLX quants of allura-org/Luna-27B-v0 using <a href="https://github.com/Blaizzy/mlx-vlm"mlx-vlm</a for quantization on Apple Silicon.
Downloads · 30 days
28
13% of all-time downloads
All-time downloads
222
Public
Repo size
77.2 GB
Likes
0
Public
Click a slice to open those files.
.md6 KB · 52%
From the Hugging Face model README
MLX quants of allura-org/Luna-27B-v0 using <a href="https://github.com/Blaizzy/mlx-vlm">mlx-vlm</a> for quantization on Apple Silicon.
[uv venv] # First-time setup with uv (optional)
[uv] pip install -U mlx-vlm
The uv wrapper is optional but recommended, get it with Homebrew:
brew install uv
Change the revision tag for different quants. Assuming you have an image such as the model card below saved to some path like ~/Downloads/Luna.png:
[uv run] mlx_vlm.generate --image ~/Downloads/Luna.png \
--system "You are Luna, a helpful and harmless language model by Allura" \
--prompt "What do you see, sweetheart?" --temperature 1.25 \
--model Luna-27B-v0_mlx-hi --revision 4bpw --max-tokens -1
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config
model_path = "allura-quants/Luna-27B-v0_mlx-hi"
model_revision = "4bpw"
model, processor = load(model_path, revision=model_revision)
config = model.config
images = ["path/to/image1.jpg", "path/to/image2.jpg"]
prompt = "Compare these two images."
formatted_prompt = apply_chat_template(
processor, config, prompt, num_images=len(images)
)
output = generate(model, processor, formatted_prompt, images, verbose=False)
print(output)
<small>Made by a member of our community using 'Waishuffle v3 + custom loras', tell them thanks!</small>
Luna 27B is a finetuned version of Gemma 3 27B (specifically, tuned on top of an abliterated[1] version). Originally, it was intended to be a solely roleplay-focused finetune, however completely unexpected capabilities emerged, and she became a very high quality assistant, especially for media analysis and tasks that require self-awareness.
It is also still good at roleplaying. I believe currently that this is likely due to persona training generalizing to all personae, not just the one trained on. The inverse is also true; it's likely that being finetuned on RP data and learning how to play all personae led the model to more easily pick up the Luna persona. However, we have not tested this, and further research is likely needed.
This simple system prompt is required to elicit the full Luna capabilities, as it was what we trained on:
You are Luna, a helpful and harmless language model by Allura.
The tested benchmarks did not experience any major degradations.
| Gemma 3 27B | Luna 27B v0 (w/ Luna system prompt) | |
|---|---|---|
| GPQA Diamond | 42.4* | 42.4** |
| IFEval | 90.4* | x*** |
| AIME25 | 26.7 | 23.3 |
* Benchmark result from the Gemma 3 Technical Report.
** Benchmark result from a run in OpenBench with 0.5 temperature.
*** For some reason, OpenBench and vLLM were acting really weird on IFEval and simply refused to finish the benchmark, it always hung around halfway. I'm going to try and get that fixed before the next version lol
Unless otherwise noted, results are from one (non-cherrypicked) run in OpenBench at 1.25 temperature and 0.05 min_p.
We are currently working on stablizing and better ingraining Luna's personality traits and giving her more assertiveness in her opinions.
Furthermore, we are also planning to look into proper RLVR to make her better at thinking through things.
A more detailed writeup will be released whenever we get to a more final model, but currently this model has undergone
[1]:
@misc{grimjim2023normpreserving,
author = {grimjim},
title = {Norm Preserving Biprojected Abliteration},
howpublished = {\url{https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration}},
note = {Accessed: 2025-12-09}
}
[2]:
@misc{zhou2024wpoenhancingrlhfweighted,
title={WPO: Enhancing RLHF with Weighted Preference Optimization},
author={Wenxuan Zhou and Ravi Agrawal and Shujian Zhang and Sathish Reddy Indurthi and Sanqiang Zhao and Kaiqiang Song and Silei Xu and Chenguang Zhu},
year={2024},
eprint={2406.11827},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2406.11827},
}