Downloads · 30 days
12
27% of all-time downloads
jinreiyu/cloned-Bonsai-27B-unpacked
cloned-Bonsai-27B-unpacked is a machine learning model from jinreiyu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
FP16 safetensors (HuggingFace format) of the 1-bit Bonsai 27B model. This repo exists for users who want to run Bonsai with stock HuggingFace tooling or frameworks that don't yet support 1-bit weights natively. The 1-…
Downloads · 30 days
12
27% of all-time downloads
All-time downloads
44
Public
Parameters
27.4B
54.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors54.7 GB · 100%
From the Hugging Face model README
FP16 safetensors (HuggingFace format) of the 1-bit Bonsai 27B model. This repo exists for users who want to run Bonsai with stock HuggingFace tooling or frameworks that don't yet support 1-bit weights natively. The 1-bit hybrid-attention kernels are currently in our forks of MLX, mlx-swift, and llama.cpp — once they land upstream, this unpacked version will no longer be needed.
We strongly recommend using the native 1-bit models instead. The 1-bit format is where all the benefits of Bonsai come from — a 14.2x memory reduction to 3.9 GB, interactive decoding on everyday laptops (44 tok/s on an M5 Pro), and the first 27B-class model that runs on a phone (11 tok/s on iPhone 17 Pro Max). This unpacked FP16 version is full-size (~54 GB) and does not provide any of those advantages.
For the optimized 1-bit release models (recommended):
For the quality-oriented variant: