Downloads · 30 days
21
5% of all-time downloads
whoy/GigaChat3-10B-A1.8B-bf16-gguf-unstable
GigaChat3-10B-A1.8B-bf16-gguf-unstable is a machine learning model from whoy. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as mit.
⚠️ UNSTABLE BUILD - This is an experimental GGUF conversion with known quality issues. Use for testing only.
Downloads · 30 days
21
5% of all-time downloads
All-time downloads
459
Public
Repo size
21.5 GB
Likes
1
Public
Click a slice to open those files.
.gguf21.5 GB · 100%
From the Hugging Face model README
⚠️ UNSTABLE BUILD - This is an experimental GGUF conversion with known quality issues. Use for testing only.
UPDATE - Currently, does not work with llama.cpp release b7127 and higher. Only releases below.
Experimental GGUF conversion of GigaChat3-10B-A1.8B - a Russian dialogue model with MoE + MLA architecture.
Model specs:
This conversion has degraded quality compared to the original model due to architectural incompatibility:
Why it still loads: We emulated missing MLA components using Identity matrices, which satisfies llama.cpp's loader but breaks positional logic.
✅ Good for:
❌ Bad for:
# 1. Restructure weights to emulate MLA
# Original: Q = X @ q_proj [6144, 1536]
# Emulated: Q = ((X @ Identity[1536,1536]) * ones) @ q_proj[6144,1536]
# 2. Convert with q_lora_rank = 1536
python prepare_weights.py # Creates fake q_a_proj, q_a_norm, q_b_proj
python convert_hf_to_gguf.py ./model-fixed --outfile model.gguf
Math is preserved, but RoPE positioning is broken.
# llama.cpp
./llama-cli -m model.gguf \
--temp 0.3 --top-p 0.9 -n 512 \
-p "User: [query]\nAssistant:"
# Recommended params
temperature: 0.0-0.5
top_p: 0.8-0.9
max_tokens: < 512 (quality degrades further out)
For production quality, use the original model with:
Or wait for proper llama.cpp support (requires C++ patch).
Problem: llama.cpp DeepSeek implementation assumes Q-vectors are compressed (q_lora_rank < hidden_size). GigaChat3 skips Q-compression.
Hack: Set q_lora_rank = hidden_size (1536) and inject Identity matrices to fake compression.
Result: Loader accepts it, but RoPE gets applied to wrong intermediate representation → broken positional encoding → quality loss.
If you're a llama.cpp dev: The fix is adding a branch for q_lora_rank == null in the DeepSeek V3 \ V2 attention code (~100 LOC). Happy to help test!
MIT (inherited from base model)