Downloads · 30 days
61
9% of all-time downloads
cloudbjorn/Mistral-Medium-3.5-128B-Eschaton-Uncensored-GGUF
Mistral-Medium-3.5-128B-Eschaton-Uncensored-GGUF is a text generation model from cloudbjorn. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as other.
This repository contains a Q4KM GGUF quantization of cloudbjorn/Mistral-Medium-3.5-128B-Eschaton-Uncensored for inference with llama.cpp and compatible applications.
Downloads · 30 days
61
9% of all-time downloads
All-time downloads
671
Public
Repo size
74.9 GB
Likes
0
Public
Click a slice to open those files.
.gguf74.9 GB · 100%
From the Hugging Face model README
This repository contains a Q4_K_M GGUF quantization of cloudbjorn/Mistral-Medium-3.5-128B-Eschaton-Uncensored for inference with llama.cpp and compatible applications.
The source model is a merged BF16 fine-tune of Mistral Medium 3.5 128B. Fine-tuning loaded the full-BF16 axolotl-ai-co/Mistral-Medium-3.5-128B-BF16 checkpoint and used cloudbjorn/eschaton-uncensored with the Eschaton Engine.
The quantized model file is:
Mistral-Medium-3.5-128B-Eschaton-Uncensored-Q4_K_M.gguf
Download it with the Hugging Face CLI:
hf download \
cloudbjorn/Mistral-Medium-3.5-128B-Eschaton-Uncensored-GGUF \
Mistral-Medium-3.5-128B-Eschaton-Uncensored-Q4_K_M.gguf \
--local-dir .
llama-cli \
-m Mistral-Medium-3.5-128B-Eschaton-Uncensored-Q4_K_M.gguf \
-cnv \
-ngl 999 \
-c 8192
Adjust -ngl to control GPU offload and -c to choose the runtime context allocation supported by your available memory. This is a 128B model, so substantial combined RAM and VRAM are still required even at Q4_K_M.
| Parameter | Value |
|---|---|
| Immediate source | cloudbjorn/Mistral-Medium-3.5-128B-Eschaton-Uncensored |
| Source precision | BF16 |
| Format | GGUF |
| Quantization | Q4_K_M |
| Conversion tool | llama.cpp |
| Intended runtime | llama.cpp and GGUF-compatible applications |
Q4_K_M uses mixed 4-bit quantization to reduce storage and inference memory substantially compared with the merged BF16 release. It is lossy and may differ slightly in output quality from the BF16 model.
The GGUF was created after the LoRA adapter was merged into the BF16 base model. The training run used 4-bit NF4 QLoRA with BF16 compute; the published GGUF is a separate post-training inference quantization and was not used as a training checkpoint.
| Parameter | Value |
|---|---|
| Training base | axolotl-ai-co/Mistral-Medium-3.5-128B-BF16 |
| Dataset | cloudbjorn/eschaton-uncensored |
| Framework | Eschaton Engine using Transformers, TRL, PEFT and bitsandbytes |
| Training method | 4-bit NF4 QLoRA with BF16 compute |
| Epochs | 1 |
| Training sequence length | 2,048 tokens |
| LoRA rank / alpha | 32 / 64 |
| LoRA targets | all-linear language-model modules; vision and multimodal-projector modules excluded |
| Micro-batch / gradient accumulation | 1 / 32 |
| Effective batch size | 32 |
| Learning rate | 5e-6 |
| Optimizer | 8-bit paged AdamW |
| Scheduler | Linear with 50 warmup steps |
| Weight decay | 0.01 |
| Seed | 3407 |
The fine-tune focuses on direct, neutral, and useful responses to sensitive, gritty, controversial, emotionally intimate, and technically demanding prompts without repetitive moralizing or canned disclaimers.
No standardized benchmark results are reported for this fine-tune or its Q4_K_M quantization. Users should evaluate it against their own instruction-following, reasoning, coding, safety, and domain-specific requirements before deployment.
This derivative follows the base model's Modified MIT License. Review that license and the upstream model card before use or redistribution.