Downloads · 30 days
21
2% of all-time downloads
rakisis-core/Gemma-4-26B-A4B-Q4K_M-GGUF
Gemma-4-26B-A4B-Q4K_M-GGUF is a machine learning model from rakisis-core. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as gemma.
language: - en - ko basemodel: google/gemma-4-26b-a4b tags: - amd - rocm - gguf - llama.cpp - xinkang-labs - resonance-agi - high-performance modelcreator: XinXin-Kang --- "Quantized on consumer AMD GPU (RX 7900 XTX /…
Downloads · 30 days
21
2% of all-time downloads
All-time downloads
1.3K
Public
Repo size
16.8 GB
Likes
1
Public
Click a slice to open those files.
.gguf16.8 GB · 100%
From the Hugging Face model README
language:
"Quantized on consumer AMD GPU (RX 7900 XTX / gfx1100) — not NVIDIA, not MI300. Proof that AMD RX series works! 🔴"
This model is a personal quantization for research purposes, part of our ongoing exploration of resonance-based intelligence at Xinkang Labs. I am sharing this to support the AMD GPU community and to prove that high-performance LLM inference is fully achievable on consumer Radeon hardware.
CLI (Terminal):
llama-cli -m gemma-4-xinxin-kang-26B-A4B-q4.gguf -ngl 99 -c 8192 -cnv
Server (Open WebUI / API):
HIP_VISIBLE_DEVICES=0 llama-server -m gemma-4-xinxin-kang-26B-A4B-q4.gguf -ngl 99 --host 0.0.0.0 --port 8000
-ngl 99offloads all layers to GPU. Adjust-cfor context length based on your VRAM.
Xinkang Labs is an independent AI research lab exploring the frontier of resonance-based intelligence and local LLM deployment.
👉 Xinkang Labs Official Website
This project was made possible with the assistance of:
Human + AI collaboration at its best. 🤝
— XinXin-Kang / Xinkang Labs 🌐 xinkanglabs.com.au
I hope this contributes even in a small way to your local AI experience. Any feedback is welcome!
This GGUF model was specifically quantized and verified on a consumer AMD GPU stack. If you want to achieve the absolute maximum performance (e.g., ~102 t/s Token Generation) and need the exact container deployment scripts, please check out the official repository:
Included in the Repository:
rocm/pytorch)llama.cpp (GGML_HIP=ON) optimal compilation flagsvLLM server startup automation scripts이 모델은 Xinkang Labs에서 진행 중인 공명 기반 지능 연구의 일환으로 테스트되었습니다. 전문적인 배포용은 아니지만, 비슷한 사양의 AMD GPU 유저분들께 작은 도움이 되었으면 하는 마음으로 공유합니다. 순정 ROCm 환경에서 안정적인 동작과 강력한 성능을 확인하였습니다.
CLI (터미널):
llama-cli -m gemma-4-xinxin-kang-26B-A4B-q4.gguf -ngl 99 -c 8192 -cnv
서버 모드 (Open WebUI 연동):
HIP_VISIBLE_DEVICES=0 llama-server -m gemma-4-xinxin-kang-26B-A4B-q4.gguf -ngl 99 --host 0.0.0.0 --port 8000
AMD GPU 사용 시
HIP_VISIBLE_DEVICES=0를 꼭 붙여주세요.
gemma-4-xinxin-kang-26B-A4B-q4.gguf본 프로젝트는 다음 AI들의 도움으로 완성되었습니다:
— XinXin-Kang / Xinkang Labs 🌐 xinkanglabs.com.au
I hope this contributes even in a small way to your local AI experience. Any feedback is welcome!
docker run -it \
--device=/dev/kfd \
--device=/dev/dri/card0 \
--device=/dev/dri/card1 \
--device=/dev/dri/renderD128 \
--device=/dev/dri/renderD129 \
--group-add video \
-v /home/$USER/dojoka:/workspace \
-p 8000:8000 \
rocm/pytorch:latest bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS="gfx1100" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=/opt/rocm/bin/hipcc \
-DCMAKE_CXX_COMPILER=/opt/rocm/bin/hipcc \
-DCMAKE_PREFIX_PATH=/opt/rocm-7.2.3
cmake --build build --config Release -j$(nproc)
HIP_VISIBLE_DEVICES=0 ./build/bin/llama-server \
-m /workspace/gemma-4-xinxin-kang-26B-A4B-q4.gguf \
-ngl 99 \
--host 0.0.0.0 \
--port 8000
HIP_VISIBLE_DEVICES=0 ./build/bin/llama-cli \
-m /workspace/gemma-4-xinxin-kang-26B-A4B-q4.gguf \
-ngl 99 -cnv
HIP_VISIBLE_DEVICES=0 — GPU 0만 사용 (CPU iGPU 충돌 방지)-ngl 99 — 모든 레이어를 GPU에 올림 (필수!)-DCMAKE_PREFIX_PATH=/opt/rocm-7.2.3 — ROCm 버전 경로 확인 필요| GPU 모델 | gfx 코드 |
|---|---|
| RX 7900 XTX/XT | gfx1100 |
| RX 7800 XT / 7700 XT | gfx1101 |
| RX 6900/6800 XT | gfx1030 |
| RX 6700 XT | gfx1031 |