Downloads · 30 days
155
7% of all-time downloads
pinkelephantlimited/pinkelephant-llm-48b-s-gguf
pinkelephant-llm-48b-s-gguf is a text generation model from pinkelephantlimited. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as mit.
Downloads · 30 days
155
7% of all-time downloads
All-time downloads
2.2K
Public
Repo size
162 GB
Likes
0
Public
Click a slice to open those files.
.gguf37.7 GB · 100%
From the Hugging Face model README
From a single dense core to an SFT + DPO–aligned sparse Mixture-of-Experts flagship — converted, quantized, and Ollama-ready so you can run the entire success story locally.
Pink Elephant 48B-S is the culmination of a six-stage lineage: a dense 14B foundation, upcycled into an 8-expert sparse MoE, refined through a second training epoch (48B-S), aligned with supervised fine-tuning and direct preference optimization, then merged bit-exact and converted to GGUF — ready to run in Ollama / llama.cpp on a single consumer GPU.
| What you get | Why it matters |
|---|---|
| 🧠 47.7B total params — only ~14.66B active / token (30.7%) | Dense-48B quality at a sparse fraction of the compute |
| ⚡ 3.2× fewer FLOPs vs. a dense 48B | Faster inference on the hardware you already own |
| 🎯 8 experts, top-2 routing | Specialized, focused reasoning per token |
| 🛡️ SFT + DPO aligned | Predictable, helpful, preference-tuned behavior |
| 📦 Q4_K_M quantized (8.3 GB) | Fits on an 8–12 GB consumer GPU |
| 🚀 Ollama-ready | One command from zero to running |
| 🆓 MIT license | Free for commercial use & research |
<p align="center"> <img src="figures/benchmarks.png" alt="Aligned 48B-S benchmark highlights" width="96%"> </p>✅ Tested — corrected build runs clean
We download the corrected Q4_K_M GGUF (
pe_48b_s_dpo_fix_Q4_K_M.gguf) fresh from this repo and ran it end-to-end in llama.cpp / Ollama: it offloads to GPU, and produces coherent, on-topic, non-degenerate output. (The earlier 8-expert build lost the router bias and could garble; the shipped files are the corrected 2-expert build — seedocs/VERIFICATION.mdandtools/conversion/pinkelephant.py.) See the 📦 GGUF files table for exactly which hardware each file needs.
Verified aligned benchmarks (bf16, greedy, single NVIDIA RTX PRO 6000 Blackwell — our own infrastructure):
🚀 Start here for clean output:
docs/HOW_TO_RUN.md— correct ChatML template + sampler settings. 📘 Read the whole story:docs/ULTIMATE_REPORT.md— full traceability for investors, engineers, and users.
Pretraining a 48B model from scratch is expensive — and wasteful, because most of that compute goes to reconstructing what a good dense 14B already knows. Pink Elephant took the smarter route:
The result: more capability per flop, per watt, per dollar — on hardware you can actually afford to own.
One lineage, six milestones: a proven dense core → upcycled to sparse scale → refined → supervised fine-tuned → preference-aligned → bit-exact merged and converted to GGUF.
A dense 14.66B decoder-only model, instruction-tuned on 99,661 curated code +
mathematics examples (bigcode/self-oss-instruct + open-r1/OpenR1-Math). It proved the
thesis:
Rather than train 48B from scratch, Pink Elephant upcycled the 14B foundation into an 8-expert Mixture-of-Experts — feedforward weights split and replicated into eight complementary experts per layer, exactly preserving the parent's behavior at initialization. A sparse brain with dense-scale capacity at ~30.7% activation per token.
<p align="center"> <img src="figures/architecture.png" alt="48B-S MoE architecture" width="96%"> </p>A second full epoch (3,114 steps) on the same curriculum with a freshly restarted cosine schedule. Training loss fell 32% (0.0616 → 0.0419) with validation loss steady near 0.50 — refinement, not overfitting.
<p align="center"> <img src="figures/48bs_training_loss.png" alt="48B-S training loss" width="62%"> </p>An industrial-standard QLoRA recipe (frozen 4-bit NF4 base purely as a memory-efficient training vehicle; all math in native bf16; adapters full precision) fine-tuned the whole 47.7B on a single GPU. Training loss fell 0.65 → ~0.44; held-out CE identified step 90,000 as the best adapter (CE 0.5392).
| <img src="figures/sft_loss.png" width="340"><br>SFT training loss | <img src="figures/sft_heldout.png" width="340"><br>SFT held-out evaluation → best at step 90k |
An alignment campaign using a frozen reference + trainable policy (QLoRA-for-DPO). The defining engineering decision: an evidence-based turning-point early stop — we watched the loss curve, found its true minimum 0.4763 at step 11,579, then stopped 1,500 steps later. A clean 2,000-pair held-out preference set shows ~0.65 preference accuracy.
<p align="center"> <img src="figures/dpo_loss.png" alt="DPO training loss with min-loss turning point" width="62%"> </p>
pinkelephant-llm-48b-s-gguf— you are here
We fused the best SFT adapter (step 90,000) + best DPO adapter (min-loss step 11,579) into the 48B-S base — bit-exact — then converted the non-standard fused-layout MoE to standard GGUF and quantized it.
base → moe → train → sft → dpo → verification → release
<p align="center">
<img src="figures/parameters.png" alt="48B-S parameter summary" width="96%">
</p>
Because it is a compact, aligned, MIT-licensed sparse MoE that fits on consumer hardware, Pink Elephant 48B-S GGUF is ideal for:
| Scenario | Why it fits |
|---|---|
| 🧑💻 Local code & math copilot | Trained on code + math; strong HumanEval / GSM8K; runs offline |
| 🔒 Private / on-prem deployment | 8.3 GB model — no data leaves your machine |
| 🏭 Edge & cost-sensitive serving | 3.2× fewer FLOPs than dense 48B; 30.7% activation saves power/cost |
| 📚 Research & experimentation | MIT license, fully reproducible lineage & verification |
| 🎓 Education / demos | One-command Ollama bring-up; the whole story is public & measured |
| Specification | Value |
|---|---|
| Total parameters | 47,691,290,048 (47.7B) |
| Active params / token | ~14.66B (30.7%) |
| Sparse compute advantage | 3.2× fewer FLOPs vs. dense 48B |
| Architecture | MoE · 8 experts · top-2 routing |
| Transformer layers / hidden / expert | 40 · 5,120 / 8,960 |
| Attention heads | 40 (10 KV heads, head dim 128) |
| Context window | 16,384 tokens |
| Vocabulary | 100,352 |
| Source precision | Native bfloat16 |
| Merge formula | W = W_base + 2·(SFT_B@SFT_A) + 2·(DPO_B@DPO_A) (r=32, α=64) |
| License | MIT — free for commercial use |
✅ Tested (corrected build): the Q4_K_M file below (
pe_48b_s_dpo_fix_Q4_K_M.gguf) was downloaded fresh from this repo and run end-to-end in llama.cpp / Ollama — created as a model, offloaded to GPU, and produced coherent, non-degenerate output. It is the corrected 2-expert build (the earlier 8-expert build dropped the router bias and could garble; superseded). Seedocs/VERIFICATION.mdandtools/conversion/pinkelephant.pyfor the fix.
| File | Quant | Size | Minimum hardware to run it | Best experience |
|---|---|---|---|---|
pe_48b_s_dpo_fix_Q4_K_M.gguf | Q4_K_M | 8.3 GB | ≥ 16 GB RAM (CPU-only) or GPU with ≥ 8 GB VRAM for full offload | 8–12 GB GPU (full offload) for interactive speed; the recommended file for Ollama |
pe_48b_s_dpo_fix.gguf | f16 (BF16) | 29.3 GB | ≥ 32 GB unified memory / VRAM (e.g. 32 GB or 2×16 GB GPU) | Full-precision reference for servers; needs a 32 GB-class host |
How to read this:
Note on quality/speed and size: these are the corrected 2-expert (effective) build of the SFT+DPO-aligned 48B-S. It ships only the live experts {0,4} (the 8-expert source has 6 dead experts that were never routed-to), which is why the files are far smaller than the old 8-expert GGUF. Q4_K_M trades a small amount of precision for a much smaller file; f16 preserves full fidelity. Both fix the router-bias bug from the earlier release.
Install Ollama from https://ollama.com and confirm it's running:
ollama --version
ollama pull hf.co/pinkelephantlimited/pinkelephant-llm-48b-s-gguf:Q4_K_M
The repo ships a ready-made Modelfile with the correct ChatML template and safe sampler
settings (this matters — see ⚙️ Sampler & format notes below):
FROM ./pe_48b_s_dpo_fix_Q4_K_M.gguf
# Lower temperature + repeat_penalty 1.0 avoid degenerate/garbled output on this MoE.
PARAMETER temperature 0.3
PARAMETER repeat_penalty 1.0
PARAMETER top_p 0.9
PARAMETER num_ctx 4096
# ChatML (OpenHermes-2.5) template — REQUIRED for correct output.
TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
Download pe_48b_s_dpo_fix_Q4_K_M.gguf from the Files tab into this folder, then:
ollama create pe-48b -f Modelfile
ollama run pe-48b "Write a Python function that checks if a number is prime."
curl http://localhost:11434/api/chat -d '{
"model": "pe-48b",
"messages": [{"role": "user", "content": "What is the capital of France?"}],
"stream": false
}'
Python:
import requests
r = requests.post("http://localhost:11434/api/chat", json={
"model": "pe-48b",
"messages": [{"role": "user", "content": "Explain MoE in one sentence."}],
"stream": False,
})
print(r.json()["message"]["content"])
This model is ChatML / OpenHermes-2.5 aligned (SFT + DPO). Getting clean output depends on two things: the chat template, and conservative sampling.
It must be prompted in ChatML form. The bundled Modelfile already sets the correct
ChatML TEMPLATE, so plain ollama run pe-48b "..." and /api/chat are handled for you.
If you bypass the template and send a bare/raw prompt (e.g. via /api/generate), you
must wrap it yourself, exactly like this (note the closing assistant tag):
<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n
A bare prompt such as "The capital of France is" will not auto-complete correctly.
Use a low temperature and keep repeat_penalty at 1.0. The default Ollama sampler
(temperature ~0.7, repeat_penalty ~1.1) makes this model garble / repeat words.
The bundled Modelfile pins temperature 0.3, repeat_penalty 1.0, top_p 0.9, which
gives stable, non-degenerate output. For the most deterministic results you may set
temperature 0. Do not raise temperature much above 0.3 without re-testing.
Known robustness note (verified). The raw-completion path with an explicit ChatML
prompt (point 1) is the most reliable on this custom MoE — it produced the model's
exact correct answer deterministically. The Ollama /api/chat template path generally
works, but in some Ollama versions it can be less stable for this architecture; if you
ever see odd output via /api/chat, switch to the raw-completion form in point 1 with
"temperature": 0 (deterministic) as a reliable fallback.
Tip: prefer the chat template (or the explicit ChatML raw-completion form) rather than bare completion for natural, instruction-following output from this SFT/DPO-aligned model.
| Path | What it is |
|---|---|
docs/BUILD_REPORT.md | Full methodology: merge, convert, quantize, verify |
tools/conversion/pinkelephant.py | Custom converter — splits fused QKV, splits gate/up, stacks 8 experts → LLAMA arch |
tools/pe_init.patch | One-line registry hook into llama.cpp |
tools/make_mini_model.py | Cheap mini-model rig that proved the converter |
scripts/merge_48b.py | Bit-exact SFT+DPO merge (max|diff| = 0.0 vs. recomputed) |
scripts/verify_merge.py | Independent merge sanity check |
scripts/run_convert.sh / quant_real.sh | GGUF conversion + Q4_K_M quantization |
See docs/VERIFICATION.md for the complete, measurement-by-measurement evidence: bit-exact merge, all-403-tensor integrity, router top-2 routing, conversion fidelity, and the HF → Ollama round-trip.
merged == base + 2·SFT + 2·DPO with max|diff| = 0.000e+00.ffn_{gate,up,down}_exps[·,·,8].o_proj, norms, qkv→q match source within f16 rounding.| Stage | Repo |
|---|---|
| 🐘 Dense 14B core | pink-elephant-llm-14b |
| 🔀 48B MoE (epoch-1) | pinkelephant-llm-48b |
| 💎 48B-S base | pinkelephant-llm-48b-s |
| 🎓 SFT adapters | pinkelephant-llm-48b-s-sft |
| 🛡️ DPO adapters | pinkelephant-llm-48b-s-dpo |
| 🚀 GGUF (this repo) | pinkelephant-llm-48b-s-gguf |
# Fastest path — pull straight from Hugging Face
ollama pull hf.co/pinkelephantlimited/pinkelephant-llm-48b-s-gguf:Q4_K_M
ollama run pinkelephant-llm-48b-s-gguf "Explain mixture of experts like I'm five."
Or watch the whole story come alive in the Ultimate Project Report.
MIT License — free for commercial and research use, modification, and redistribution. Use at your own risk.
<div align="center">🦾 Pink Elephant Limited · Pink Elephant 48B-S GGUF · 2026 — from a 14B dense core to a runnable, aligned MoE flagship, in one lineage.
</div>