Downloads · 30 days
690
6% of all-time downloads
CelesteImperia/Gemma-4-31B-Dense-Platinum-GGUF
Gemma-4-31B-Dense-Platinum-GGUF is a machine learning model from CelesteImperia. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
Downloads · 30 days
690
6% of all-time downloads
All-time downloads
12.3K
Public
Parameters
32.7B
176 GB on disk
Likes
4
Public
Click a slice to open those files.
.gguf114 GB · 64%
From the Hugging Face model README
Optimized GGUF weights for Gemma 4 (31B Dense), ported by CelesteImperia on an NVIDIA RTX 3090 AI Workstation.
llama.cpp gemma4-day0 branch (b8642).This repository contains the Platinum Series universal GGUF release of Gemma-4-31B-Dense. This collection provides professional-grade quantization levels optimized for high-fidelity reasoning, long-context retrieval, and multi-step logic. Ported manually to ensure zero weight-map corruption, these quants are optimized for local 24GB VRAM workstations.
| File | Method | Description |
|---|---|---|
| Q3_K_M | k-quant | The Gold Standard. Consumer Grade. (~14.2 GB) Optimized for 16GB VRAM cards (RTX 4080 / A4000). |
| Q4_K_M | k-quant | The Gold Standard. Optimal balance of logic retention and inference speed. |
| Q5_K_M | k-quant | Platinum Tier. Recommended for the RTX 3090 to maintain high reasoning stability. |
| Q6_K | k-quant | High-bit precision for complex logic and massive 100k+ token document analysis. |
| Q8_0 | block-quant | The "Reference" version. Near-perfect fidelity to the original BF16 master. |
To use the native Thinking Mode, ensure you use the correct control tokens:
ollama run Celeste-Gemma-4-31B-Q4_K_M
To run these engines using the provided python script :
from llama_cpp import Llama
# Initialize the model for 24GB VRAM (RTX 3090)
llm = Llama(
model_path="./Gemma-4-31B-Q4_K_M.gguf",
n_gpu_layers=-1, # Offload all layers to VRAM
n_ctx=32768, # Extended context window
)
# Generate response with Native Thinking tokens
output = llm(
"<|think|>\nAnalyze the logic of the following legal document:",
max_tokens=1024,
stop=["<turn|>", "<|file_separator|>"],
echo=True
)
print(output['choices'][0]['text'])
This collection is fully compatible with .NET applications via the csharp script and the LLamaSharp library.
using LLama.Common;
using LLama;
var parameters = new ModelParams("Gemma-4-31B-Q4_K_M.gguf")
{
ContextSize = 32768,
GpuLayerCount = -1 // Utilize all available CUDA cores on RTX 3090
};
using var weights = LLamaWeights.LoadFromFile(parameters);
using var context = weights.CreateContext(parameters);
var executor = new InteractiveExecutor(context);
var chatHistory = new ChatHistory();
chatHistory.AddMessage(AuthorRole.System, "You are a helpful assistant.");
var session = new ChatSession(executor, chatHistory);
await foreach (var text in session.ChatAsync(new ChatHistory.Message(AuthorRole.User, "Explain GST impact on small businesses."), new InferenceParams { MaxTokens = 1024 }))
{
Console.Write(text);
}
Given the 31B parameter count and the 256K context architecture, the following configurations are recommended:
Maintaining the production line for high-fidelity models requires significant hardware resources. If these tools power your research or industrial projects, please consider supporting the development:
| Platform | Support Link |
|---|---|
| Global & India | Support via Razorpay |
Scan to support via UPI (India Only):
<img src="https://huggingface.co/datasets/CelesteImperia/Assets/resolve/main/QrCode.jpeg" width="200">Connect with the architect: Abhishek Jaiswal on LinkedIn