Downloads · 30 days
901
5% of all-time downloads
Mungert/Dans-PersonalityEngine-V1.3.0-24b-GGUF
Dans-PersonalityEngine-V1.3.0-24b-GGUF is a text generation model from Mungert. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model was generated using llama.cpp at commit f5cd27b7.
Downloads · 30 days
901
5% of all-time downloads
All-time downloads
19.9K
Public
Repo size
428 GB
Likes
8
Public
Click a slice to open those files.
.gguf428 GB · 100%
From the Hugging Face model README
This model was generated using llama.cpp at commit f5cd27b7.
Our latest quantization method introduces precision-adaptive quantization for ultra-low-bit models (1-2 bit), with benchmark-proven improvements on Llama-3-8B. This approach uses layer-specific strategies to preserve accuracy while maintaining extreme memory efficiency.
All tests conducted on Llama-3-8B-Instruct using:
| Quantization | Standard PPL | DynamicGate PPL | Δ PPL | Std Size | DG Size | Δ Size | Std Speed | DG Speed |
|---|---|---|---|---|---|---|---|---|
| IQ2_XXS | 11.30 | 9.84 | -12.9% | 2.5G | 2.6G | +0.1G | 234s | 246s |
| IQ2_XS | 11.72 | 11.63 | -0.8% | 2.7G | 2.8G | +0.1G | 242s | 246s |
| IQ2_S | 14.31 | 9.02 | -36.9% | 2.7G | 2.9G | +0.2G | 238s | 244s |
| IQ1_M | 27.46 | 15.41 | -43.9% | 2.2G | 2.5G | +0.3G | 206s | 212s |
| IQ1_S | 53.07 | 32.00 | -39.7% | 2.1G | 2.4G | +0.3G | 184s | 209s |
Key:
Key Improvements:
Tradeoffs:
📌 Fitting models into GPU VRAM
✔ Memory-constrained deployments
✔ Cpu and Edge Devices where 1-2bit errors can be tolerated
✔ Research into ultra-low-bit quantization
Selecting the correct model format depends on your hardware capabilities and memory constraints.
📌 Use BF16 if:
✔ Your hardware has native BF16 support (e.g., newer GPUs, TPUs).
✔ You want higher precision while saving memory.
✔ You plan to requantize the model into another format.
📌 Avoid BF16 if:
❌ Your hardware does not support BF16 (it may fall back to FP32 and run slower).
❌ You need compatibility with older devices that lack BF16 optimization.
📌 Use F16 if:
✔ Your hardware supports FP16 but not BF16.
✔ You need a balance between speed, memory usage, and accuracy.
✔ You are running on a GPU or another device optimized for FP16 computations.
📌 Avoid F16 if:
❌ Your device lacks native FP16 support (it may run slower than expected).
❌ You have memory limitations.
Quantization reduces model size and memory usage while maintaining as much accuracy as possible.
📌 Use Quantized Models if:
✔ You are running inference on a CPU and need an optimized model.
✔ Your device has low VRAM and cannot load full-precision models.
✔ You want to reduce memory footprint while keeping reasonable accuracy.
📌 Avoid Quantized Models if:
❌ You need maximum accuracy (full-precision models are better for this).
❌ Your hardware has enough VRAM for higher-precision formats (BF16/F16).
These models are optimized for extreme memory efficiency, making them ideal for low-power devices or large-scale deployments where memory is a critical constraint.
IQ3_XS: Ultra-low-bit quantization (3-bit) with extreme memory efficiency.
IQ3_S: Small block size for maximum memory efficiency.
IQ3_M: Medium block size for better accuracy than IQ3_S.
Q4_K: 4-bit quantization with block-wise optimization for better accuracy.
Q4_0: Pure 4-bit quantization, optimized for ARM devices.
| Model Format | Precision | Memory Usage | Device Requirements | Best Use Case |
|---|---|---|---|---|
| BF16 | Highest | High | BF16-supported GPU/CPUs | High-speed inference with reduced memory |
| F16 | High | High | FP16-supported devices | GPU inference when BF16 isn't available |
| Q4_K | Medium Low | Low | CPU or Low-VRAM devices | Best for memory-constrained environments |
| Q6_K | Medium | Moderate | CPU with more memory | Better accuracy while still being quantized |
| Q8_0 | High | Moderate | CPU or GPU with enough VRAM | Best accuracy among quantized models |
| IQ3_XS | Very Low | Very Low | Ultra-low-memory devices | Extreme memory efficiency and low accuracy |
| Q4_0 | Low | Low | ARM or low-memory devices | llama.cpp can optimize for ARM devices |
Dans-PersonalityEngine-V1.3.0-24b-bf16.ggufDans-PersonalityEngine-V1.3.0-24b-f16.ggufDans-PersonalityEngine-V1.3.0-24b-bf16-q8_0.ggufDans-PersonalityEngine-V1.3.0-24b-f16-q8_0.ggufDans-PersonalityEngine-V1.3.0-24b-q4_k.ggufDans-PersonalityEngine-V1.3.0-24b-q4_k_s.ggufDans-PersonalityEngine-V1.3.0-24b-q6_k.ggufDans-PersonalityEngine-V1.3.0-24b-q8_0.ggufDans-PersonalityEngine-V1.3.0-24b-iq3_xs.ggufDans-PersonalityEngine-V1.3.0-24b-iq3_m.ggufDans-PersonalityEngine-V1.3.0-24b-q4_0.gguf❤ Please click "Like" if you find this useful!
Help me test my AI-Powered Network Monitor Assistant with quantum-ready security checks:
👉 Quantum Network Monitor
💬 How to test:
Choose an AI assistant type:
TurboLLM (GPT-4o-mini)HugLLM (Hugginface Open-source)TestLLM (Experimental CPU-only)I’m pushing the limits of small open-source models for AI network monitoring, specifically:
🟡 TestLLM – Current experimental model (llama.cpp on 2 CPU threads):
🟢 TurboLLM – Uses gpt-4o-mini for:
🔵 HugLLM – Latest Open-source models:
"Give me info on my websites SSL certificate""Check if my server is using quantum safe encyption for communication""Run a comprehensive security audit on my server"I fund the servers used to create these model files, run the Quantum Network Monitor service, and pay for inference from Novita and OpenAI—all out of my own pocket. All the code behind the model creation and the Quantum Network Monitor project is open source. Feel free to use whatever you find helpful.
If you appreciate the work, please consider buying me a coffee ☕. Your support helps cover service costs and allows me to raise token limits for everyone.
I'm also open to job opportunities or sponsorship.
Thank you! 😊
<!doctype html> <html lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>Dans-PersonalityEngine-V1.3.0-24b</title> </head> <div class="crt-container"> <div class="crt-case"> <div class="crt-inner-case"> <div class="crt-bezel"> <div class="terminal-screen"> <div style="text-align: center"> <h2>Dans-PersonalityEngine-V1.3.0-24b</h2> <pre class="code-block" style="display: inline-block; text-align: left; font-size: clamp(2px, 0.8vw, 14px); line-height: 1.2; max-width: 100%; overflow: hidden; white-space: pre;"> ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⠀⠄⠀⡂⠀⠁⡄⢀⠁⢀⣈⡄⠌⠐⠠⠤⠄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⡄⠆⠀⢠⠀⠛⣸⣄⣶⣾⡷⡾⠘⠃⢀⠀⣴⠀⡄⠰⢆⣠⠘⠰⠀⡀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠃⠀⡋⢀⣤⡿⠟⠋⠁⠀⡠⠤⢇⠋⠀⠈⠃⢀⠀⠈⡡⠤⠀⠀⠁⢄⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠁⡂⠀⠀⣀⣔⣧⠟⠋⠀⢀⡄⠀⠪⣀⡂⢁⠛⢆⠀⠀⠀⢎⢀⠄⢡⠢⠛⠠⡀⠀⠄⠀⠀ ⠀⠀⡀⠡⢑⠌⠈⣧⣮⢾⢏⠁⠀⠀⡀⠠⠦⠈⠀⠞⠑⠁⠀⠀⢧⡄⠈⡜⠷⠒⢸⡇⠐⠇⠿⠈⣖⠂⠀ ⠀⢌⠀⠤⠀⢠⣞⣾⡗⠁⠀⠈⠁⢨⡼⠀⠀⠀⢀⠀⣀⡤⣄⠄⠈⢻⡇⠀⠐⣠⠜⠑⠁⠀⣀⡔⡿⠨⡄ ⠈⠂⠀⠆⠀⣼⣾⠟⠀⠑⠀⡐⠗⠉⠀⠐⠶⣤⡵⠋⠀⠠⠹⡌⡀⠘⠇⢠⣾⡣⣀⡴⠋⠅⠈⢊⠠⡱⡀ ⠪⠑⢌⠂⣼⣿⡟⠀⠀⠙⠀⠀⠀⡀⠀⠀⠐⡞⡐⠀⠀⡧⠀⢀⠠⠀⣁⠾⡇⠀⠙⡁⠀⠀⢀⣨⣄⡠⢱ ⣸⠈⠊⠙⣛⣿⡧⠔⠚⠛⠳⣄⣀⡬⠤⠬⠼⡣⠃⠀⢀⡗⠀⡤⠞⠙⠄⠂⠃⢀⣠⣤⠶⠙⠅⠁⠃⠋⠈ ⢋⠼⣀⠰⢯⢿⠁⠀⢢⠀⠀⢐⠋⡀⠀⠈⠁⠀⣀⣰⠏⠒⠙⠈⠀⣀⡤⠞⢁⣼⠏⠘⢀⣀⢤⢤⡐⢈⠂ ⠀⠢⠀⠀⠸⣿⡄⠲⠚⠘⠚⠃⢀⠀⠈⢋⠶⠛⠉⠉⢃⣀⢤⢾⠋⣁⡤⡚⠁⢹⠁⠠⢛⠠⠬⠁⢬⠀⠀ ⠀⠈⢳⣒⠋⠉⣿⢐⠠⣀⣃⠀⠀⠉⠂⢁⣀⣀⡤⢞⠩⢑⡨⠰⡞⠁⠁⢀⡠⠾⠎⡈⡌⡈⡓⡀⠄⠀⠀ ⠀⠀⠀⠉⠘⠃⢻⡒⠦⢼⣿⣛⣻⣿⡷⢄⣀⣀⣠⣴⢾⣿⣆⣡⡄⣠⣪⡿⣷⣾⣷⣧⡡⠅⣇⠍⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠙⠒⠒⠛⠛⠓⠉⢹⠀⣷⠴⣻⣽⡻⢧⢻⡿⡏⣼⢿⣻⢾⣿⣿⣿⡿⢠ ⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠂⠻⠨⠰⢋⡅⠉⣑⡇⡗⣿⢂⣸⡿⣿⣛⠿⠃⠁ ⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠳⣌⣙⣸⢧⣿⣕⣼⣇⢹⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⣸⢧⢟⢟⡟⣾⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⢰⠙⣾⡟⣻⡕⣹⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⢸⢰⡏⢠⡿⠾⠋⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⢸⠸⡇⡏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠸⢸⢸⡇⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠘⠇⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ </pre> </div> <p> Dans-PersonalityEngine is a versatile model series fine-tuned on 50+ specialized datasets, designed to excel at both creative tasks (like roleplay and co-writing) and technical challenges (such as code generation, tool use, and complex reasoning). </p> <p> V1.3.0 introduces multilingual capabilities with support for 10 languages and enhanced domain expertise across multiple fields. The primary language is still English and that is where peak performance can be expected. </p> <h3>Multilingual Support</h3> <pre class="code-block"> Arabic Chinese English French German Hindi Japanese Korean Portuguese Spanish</pre> <h3>Key Details</h3> <pre class="code-block"> BASE MODEL: mistralai/Mistral-Small-3.1-24B-Base-2503 LICENSE: apache-2.0 LANGUAGE: Multilingual with 10 supported languages CONTEXT LENGTH: 32768 tokens, 131072 with degraded recall</pre> <h3>Recommended Settings</h3> <pre class="code-block"> TEMPERATURE: 1.0 TOP_P: 0.9</pre> <h3>Prompting Format</h3> <p> The model uses the following format I'll refer to as "DanChat-2": </p> <pre class="code-block"> <|system|>system prompt<|endoftext|><|user|>Hi there!<|endoftext|><|assistant|>Hey, how can I help?<|endoftext|></pre> <h3>Why not ChatML?</h3> <p> While ChatML is a standard format for LLMs, it has limitations. DanChat-2 uses special tokens for each role, this reduces biases and helps the model adapt to different tasks more readily. </p> <h3>SillyTavern Template</h3> <p> <a href="https://huggingface.co/PocketDoc/Dans-PersonalityEngine-V1.3.0-24b/resolve/main/resources/DanChat-2.json?download=true" download target="_blank" rel="noopener noreferrer" > Download Master JSON </a> </p> <h3>Inference Provider</h3> <p> This model and others are available from ⚡Mancer AI for those interested in high quality inference without owning or renting expensive hardware. </p> <p class="mancer-button-container"> <a href="https://mancer.tech/" target="_blank" rel="noopener noreferrer" class="mancer-button" > <span class="mancer-text">mancer</span> </a> </p> <h3>Training Process</h3> <p> The model was trained using Axolotl on 8x H100 GPUs for 50 hours. The resources to train this model were provided by Prime Intellect and Kalomaze. </p> <h3>Support Development</h3> <p> Development is limited by funding and resources. To help support: </p> <p>- Contact on HF</p> <p>- Email: [email protected]</p> <p class="coffee-container"> <a href="https://www.buymeacoffee.com/visually" target="_blank" rel="noopener noreferrer" > <img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="45" width="162" /> </a> </p> </div> </div> </div> </div> </div> <style> @import url("https://fonts.googleapis.com/css2?family=Consolas&display=swap"); .crt-container { padding: 10px; max-width: 1000px; margin: 0 auto; width: 95%; } .crt-case { background: #e8d7c3; border-radius: 10px; padding: 15px; box-shadow: inset -2px -2px 5px rgba(0, 0, 0, 0.3), 2px 2px 5px rgba(0, 0, 0, 0.2); } .crt-inner-case { background: #e8d7c3; border-radius: 8px; padding: 3px; box-shadow: inset -1px -1px 4px rgba(0, 0, 0, 0.3), 1px 1px 4px rgba(0, 0, 0, 0.2); } .crt-bezel { background: linear-gradient(145deg, #1a1a1a, #2a2a2a); padding: 15px; border-radius: 5px; border: 3px solid #0a0a0a; position: relative; box-shadow: inset 0 0 20px rgba(0, 0, 0, 0.5), inset 0 0 4px rgba(0, 0, 0, 0.4), inset 2px 2px 4px rgba(255, 255, 255, 0.05), inset -2px -2px 4px rgba(0, 0, 0, 0.8), 0 0 2px rgba(0, 0, 0, 0.6), -1px -1px 4px rgba(255, 255, 255, 0.1), 1px 1px 4px rgba(0, 0, 0, 0.3); } .crt-bezel::before { content: ""; position: absolute; top: 0; left: 0; right: 0; bottom: 0; background: linear-gradient( 45deg, rgba(255, 255, 255, 0.03) 0%, rgba(255, 255, 255, 0) 40%, rgba(0, 0, 0, 0.1) 60%, rgba(0, 0, 0, 0.2) 100% ); border-radius: 3px; pointer-events: none; } .terminal-screen { background: #111112; padding: 20px; border-radius: 15px; position: relative; overflow: hidden; font-family: "Consolas", monospace; font-size: clamp(12px, 1.5vw, 16px); color: #e49b3e; line-height: 1.4; text-shadow: 0 0 2px #e49b3e; /* Removed animation: flicker 0.15s infinite; */ filter: brightness(1.1) contrast(1.1); box-shadow: inset 0 0 30px rgba(0, 0, 0, 0.9), inset 0 0 8px rgba(0, 0, 0, 0.8), 0 0 5px rgba(0, 0, 0, 0.6); max-width: 80ch; margin: 0 auto; } .terminal-screen h2, .terminal-screen h3 { font-size: clamp(16px, 2vw, 20px); margin-bottom: 1em; color: #e49b3e; } .terminal-screen pre.code-block { font-size: clamp(10px, 1.3vw, 14px); white-space: pre; /* Changed from pre-wrap to pre */ margin: 1em 0; background-color: #1a1a1a; padding: 1em; border-radius: 4px; color: #e49b3e; overflow-x: auto; /* Added to enable horizontal scrolling */ } .terminal-screen::before { content: ""; position: absolute; top: 0; left: 0; right: 0; bottom: 0; background: linear-gradient( rgba(18, 16, 16, 0) 50%, rgba(0, 0, 0, 0.25) 50% ), url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADIAAAAyBAMAAADsEZWCAAAAGFBMVEUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA4o8JoAAAAB3RSTlMAGwQIEQMYADcPzwAAACJJREFUKM9jYBgFo2AU0Beg+A8YMCLxGYZCbNQEo4BaAAD5TQiR5wU9vAAAAABJRU5ErkJggg=="); background-size: 100% 2.5px; /* Removed animation: scan 1s linear infinite; */ pointer-events: none; z-index: 2; } .terminal-screen::after { content: ""; position: absolute; top: 0; left: 0; right: 0; bottom: 0; background: radial-gradient( circle at center, rgba(17, 17, 18, 0) 0%, rgba(17, 17, 18, 0.2) 50%, rgba(17, 17, 18, 0.15) 100% ); border-radius: 20px; /* Removed animation: vignette-pulse 3s infinite; */ pointer-events: none; z-index: 1; } .terminal-screen details { margin: 1em 0; padding: 0.5em; border: 1px solid #e49b3e; border-radius: 4px; } .terminal-screen summary { cursor: pointer; font-weight: bold; margin: -0.5em; padding: 0.5em; border-bottom: 1px solid #e49b3e; color: #e49b3e; } .terminal-screen details[open] summary { margin-bottom: 0.5em; } .badge-container, .coffee-container { text-align: center; margin: 1em 0; } .badge-container img, .coffee-container img { max-width: 100%; height: auto; } .terminal-screen a { color: #e49b3e; text-decoration: underline; transition: opacity 0.2s; } .terminal-screen a:hover { opacity: 0.8; } .terminal-screen strong, .terminal-screen em { color: #f0f0f0; /* off-white color for user/system messages */ } .terminal-screen p { color: #f0f0f0; /* off-white color for assistant responses */ } .terminal-screen p, .terminal-screen li { color: #e49b3e; } .terminal-screen code, .terminal-screen kbd, .terminal-screen samp { color: #e49b3e; font-family: "Consolas", monospace; text-shadow: 0 0 2px #e49b3e; background-color: #1a1a1a; padding: 0.2em 0.4em; border-radius: 4px; } .terminal-screen pre.code-block, .terminal-screen pre { font-size: clamp(10px, 1.3vw, 14px); white-space: pre; /* Changed from pre-wrap to pre */ margin: 1em 0; background-color: #1a1a1a; padding: 1em; border-radius: 4px; color: #e49b3e; overflow-x: auto; /* Added to enable horizontal scrolling */ } .mancer-button-container { text-align: left; margin: 1em 0; } .mancer-button { display: inline-flex; align-items: center; gap: 8px; background: #1a1a1a; color: #e49b3e; padding: 15px 15px; border: 2px solid #e49b3e; border-radius: 5px; text-decoration: none !important; box-shadow: 0 0 10px rgba(228, 155, 62, 0.3); transition: all 0.3s ease; position: relative; } .mancer-text { font-family: "Consolas", monospace; font-weight: bold; font-size: 20px; text-shadow: 0 0 2px #e49b3e; line-height: 1; display: inline-block; margin-left: -4px; margin-top: -2px; } .mancer-button::before { content: "⚡"; display: inline-flex; align-items: center; justify-content: center; font-size: 20px; line-height: 1; } .mancer-button:hover { background: #2a2a2a; box-shadow: 0 0 15px rgba(228, 155, 62, 0.5); text-shadow: 0 0 4px #e49b3e; text-decoration: none !important; } </style> </html>