Downloads · 30 days
132
10% of all-time downloads
xiaol/gemma-4-e4B-hybrid-rnn-mem-rwkv-fable5-gpt5.5-v1-GGUF
gemma-4-e4B-hybrid-rnn-mem-rwkv-fable5-gpt5.5-v1-GGUF is a text generation model from xiaol. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as other.
This repository contains a Gemma4 E4B Q8 GGUF package plus an experimental RWKV-MS online-memory sidecar.
Downloads · 30 days
132
10% of all-time downloads
All-time downloads
1.3K
Public
Repo size
8.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf8.6 GB · 100%
From the Hugging Face model README
This repository contains a Gemma4 E4B Q8 GGUF package plus an experimental RWKV-MS online-memory sidecar.
The RWKV-MS memory adapter was built for the tau2 telecom/mobile-data
tool-agent experiment. The accepted sidecar was exported from the step-100
checkpoint of the formatrefresh_continue200_len192_layers0_5_qo_r8 tau2
continuation run in the online-memory checkpoint repo:
xiaol/gemma-4-e4B-hybrid-rnn-mem-rwkv-fable5-gpt5.5-v1
Despite the historical repo name, the accepted checkpoint metadata says it was
trained on deterministic tau2 telecom mobile-data rule-planner traces and a
format-refresh continuation. It was not trained on GPT-5.5-generated data. The
fable5_gpt55_data entry in the metadata is a planned upgrade note, not the
data used for this accepted checkpoint.
The base GGUF and multimodal projector are normal llama.cpp artifacts. The RWKV-MS memory file is a GGUF sidecar containing adapter tensors and metadata. Executing the sidecar requires the patched llama.cpp fork below; stock llama.cpp can parse ordinary GGUF files but does not run this RWKV-MS memory sidecar.
Patched runtime:
https://github.com/xiaol/llama.cpp-online-memory
commit: 85da0c63b Add Gemma4 RWKV-MS GGUF sidecar runtime
base upstream: ggml-org/llama.cpp 1ec44d1
| File | Size | SHA-256 |
|---|---|---|
gemma-4-E4B-it-Q8_0.gguf | 8,031,240,160 bytes | fb8f0c032de00b18c710824af3c7e5777c71e5fb60b13f13575f0a9e92ddecd0 |
mmproj-gemma-4-E4B-it-Q8_0.gguf | 559,874,528 bytes | 51d4b7fd825e4569f746b200fccc5332bf914e8ef7cbe447272ce4fec6df3db6 |
gemma-4-E4B-it-rwkv-ms-memory.gguf | 1,663,840 bytes | 0c646a776b5b12c9d3657ffd2e5e581be1eb46e858f1f404afeaa7077c02974e |
The sidecar is bound to the exact base GGUF SHA-256 above. Loading it with a different quantization or regenerated GGUF will fail with a hash mismatch.
/path/to/llama-server \
-m gemma-4-E4B-it-Q8_0.gguf \
--mmproj mmproj-gemma-4-E4B-it-Q8_0.gguf \
--alias gemma-4-e4b-it-q8 \
--host 127.0.0.1 \
--port 8080 \
--ctx-size 8192 \
--n-gpu-layers 999 \
--jinja \
--reasoning off
Use the patched fork and the constrained one-sequence mode:
/path/to/llama-server \
-m gemma-4-E4B-it-Q8_0.gguf \
--alias gemma-4-e4b-it-rwkv-ms-q8 \
--host 127.0.0.1 \
--port 18083 \
--ctx-size 8192 \
--batch-size 2 \
--ubatch-size 1 \
--n-gpu-layers 999 \
--parallel 1 \
--jinja \
--reasoning off \
--rwkv-ms-sidecar gemma-4-E4B-it-rwkv-ms-memory.gguf \
--no-cont-batching \
--no-context-shift \
--no-cache-prompt \
--no-cache-idle-slots \
--cache-ram 0 \
--ctx-checkpoints 0 \
--slot-save-path ./slots
0-5.q and o delta hooks only.num_state_heads=1.rwkv_ms_num_states=4, chunk size 1024.--parallel 1 and --ubatch-size 1.0 exact-prefix
restore checks.Validation was run locally on 2026-06-26 with the patched llama.cpp CUDA build.
Runtime health:
ok: true
base_gguf_sha256: fb8f0c032de00b18c710824af3c7e5777c71e5fb60b13f13575f0a9e92ddecd0
sidecar_base_gguf_sha256: fb8f0c032de00b18c710824af3c7e5777c71e5fb60b13f13575f0a9e92ddecd0
Raw deterministic contrast suite:
temperature=0, top_k=1, top_p=1, seed=123, n_predict=64, cache_prompt=false
8/8 raw-completion prompts produced different base vs RWKV-MS continuations.
Example:
Prompt: Hi
Base: ! I'm excited to chat with you. I'm here to help you with whatever you need. Just let me know how I can assist you today!
RWKV-MS: ! I'm excited to chat with you. What's on your mind today?
The chat-history prompt suite is also included in reports/. That suite mostly
shows semantic equivalence because both endpoints receive the same conversation
history in each request. It should not be read as proof of a memory advantage.
Tau2 training metadata from the exported sidecar:
base_model: google/gemma-4-E4B-it
selected_checkpoint: step-100
data: tau2 telecom mobile-data rule-planner traces plus format-refresh continuation
wrapped_layers: 0-5
delta_heads: q,o
rank: 8
alpha: 16
trainable_memory_params: 797,808
reported tau2 telecom 20-task screen: 14/20, pass_hat_1=0.70
reports/raw_completion_contrast_suite_20260626_225007.mdreports/raw_completion_contrast_suite_20260626_225007.jsonlreports/prompt_suite_compare_20260626_215243.mdreports/prompt_suite_base_20260626_215243.jsonlreports/prompt_suite_rwkv_ms_20260626_215243.jsonlreports/rwkv_ms_runtime_health_20260626_215243.jsonreports/rwkv_ms_memory_sidecar_manifest.jsonreports/gguf_memory_manifest.jsonreports/contrast_trace_20260626_224632.jsonlThis is an experimental research/runtime artifact. It is useful for inspecting GGUF sidecar loading, deterministic generation changes, RWKV-MS recurrent state serialization checks, and constrained llama.cpp integration behavior. It is not yet a production-fast or fully general llama.cpp memory backend.