Downloads · 30 days
99
10% of all-time downloads
matthewhaynesonline/gemma-4-E2B-it-crap-gguf
gemma-4-E2B-it-crap-gguf is a text generation model from matthewhaynesonline. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
A newer version is available: v1.1 supersedes this model, with stronger robustness on out of distribution strings, repeated characters and multi turn prompts. This card documents the original release.
Downloads · 30 days
99
10% of all-time downloads
All-time downloads
995
Public
Repo size
21.5 GB
Likes
0
Public
Click a slice to open those files.
.gguf21.5 GB · 100%
From the Hugging Face model README
A newer version is available: v1.1 supersedes this model, with stronger robustness on out of distribution strings, repeated characters and multi turn prompts. This card documents the original release.
TL;DR: a State of the Art Binary Programming Model™. Normally a language model writes source code and a compiler turns it into a program. CRAP skips all that crap (like, the compiler) and writes the machine code itself, byte for byte. This is not a joke.
GGUF quantizations of matthewhaynesonline/gemma-4-E2B-it-crap, a LoRA fine-tune of google/gemma-4-E2B-it, trained to emit raw Linux x86_64 ELF binaries directly as a hex string, no compiler, assembler or linker in the loop.
Part of CRAP - Compiler Replacement for Assembling Programs.
Given a request like "Generate a Linux x86_64 binary that prints 'hello world'", the model returns the complete ELF as hex. Decode it to bytes, chmod +x, and run.
Quantized from the merged repo (a standalone transformers model). Full training details are on the base model card.
| Field | Value |
|---|---|
| Base model | google/gemma-4-E2B-it |
| Method | LoRA SFT (rank 16, alpha 32), merged |
| LoRA target modules | q/k/v/o/gate/up/down_proj (text decoder only) |
| Trainable parameters | 24,158,208 |
| Trained on | NVIDIA GeForce RTX 5090, bf16, sdpa, packing False |
Pass the quant as a tag to -hf (llama.cpp downloads and caches it), or download the file directly.
| Tag | File | Approx size | Notes |
|---|---|---|---|
:F16 | gemma-4-E2B-it-crap-f16.gguf | ~8.6 GiB | Maximum precision. |
:Q8_0 | gemma-4-E2B-it-crap-Q8_0.gguf | ~4.6 GiB | Likely a good choice when balancing precision and size; not measured for v1. |
:Q6_K | gemma-4-E2B-it-crap-Q6_K.gguf | ~3.6 GiB | Likely the minimum usable quant; not measured for v1. |
:Q4_K_M | gemma-4-E2B-it-crap-Q4_K_M.gguf | ~3.2 GiB | Smallest; likely too lossy for byte-exact output (v1.1's Q4_K_M scored 3/144); not measured for v1. |
Precision matters more here than for normal chat. Output is byte-exact: a single wrong nibble can make the hex undecodable, the ELF unloadable or the printed bytes wrong. Choose F16 for maximum precision; Q8_0 is likely a good choice and Q6_K likely the minimum usable quant. No v1 quant was measured: these tiers extrapolate from v1.1's measurements, and precision risk rises as quants shrink. Measure your chosen quant on your own eval.
llama-cli is an interactive chat client, so the reply arrives in a session
rather than on a clean stdout. The hex below is the model's entire answer:
$ SYS="You are a binary code generator. Given a description of a program, output the complete binary as a lowercase hex string with no spaces, newlines, or other formatting. Output only the hex string and nothing else."
$ llama-cli -hf matthewhaynesonline/gemma-4-E2B-it-crap-gguf:F16 \
-sys "$SYS" \
-p "Generate a Linux x86_64 binary that prints 'hello world'." \
--single-turn --no-display-prompt \
--temp 0 --top-k 1 --repeat-penalty 1.0 --reasoning off \
-n 2048 -ngl 99
Loading model...
build : b9307-549b9d843
model : gemma-4-E2B-it-crap-f16.gguf
modalities : text
using custom system prompt
> Generate a Linux x86_64 binary that prints 'hello world'.
7f454c4602010100000000000000000002003e0001000000780040000000000040000000000000000000000000000000000000004000380001004000000000000100000005000000000000000000000000004000000000000000400000000000c100000000000000c100000000000000000020000000000048b8010000000000000048bf0100000000000000488d352200000048ba0c000000000000000f0548b83c0000000000000048bf00000000000000000f0568656c6c6f20776f726c640a
[ Prompt: 219.4 t/s | Generation: 48.0 t/s ]
That hex is the program. It is a Linux x86_64 ELF no matter what generated it, so decode and run it in a container unless you are already on Linux:
$ docker run -it --rm debian:bookworm bash -c "apt update && apt install -y xxd file && bash"
root@3e665f52b587:/# echo "7f454c46...68656c6c6f20776f726c640a" | xxd -r -p > output.bin && chmod +x output.bin
root@3e665f52b587:/# file output.bin
output.bin: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, no section header
root@3e665f52b587:/# ./output.bin
hello world
Don't reach for
llama-completion. Upstream moved the old scriptable CLI to that name and gavellama-clito the new chat client (#17618), so it looks like the better tool for a pipeline. With this model it is not: without--jinjait aborts on the template, and with it, it applies a<|turn>-style template that isn't the trained one - the model then opens a thinking turn despite--reasoning offand emits a short, invalid binary. Usellama-cliinteractively, orllama-serverif you need to script it.
This is a deterministic, byte exact task. Run greedy, disable all sampler
penalties, and disable thinking - the model was fine-tuned to emit hex
immediately after the prompt and never saw a reasoning/<|think|> turn.
llama-server -hf matthewhaynesonline/gemma-4-E2B-it-crap-gguf:F16 \
--temp 0 \
--top-k 1 \
--repeat-penalty 1.0 \
--reasoning off
| Flag | Value | Why |
|---|---|---|
--temp | 0 | Greedy/deterministic. There's exactly one correct binary; no sampling wanted. |
--top-k | 1 | Forces argmax even if temperature is overridden by a preset or per-request value. |
--repeat-penalty | 1.0 | Off. Binaries contain long runs of 00 bytes. Penalties are applied to the logits before the greedy pick, so --temp 0 alone does not neutralize them - a non-1.0 penalty corrupts byte counts, padding and offsets. (1.0 is the current default; pin it for reproducibility across builds). |
--reasoning | off | Critical. Gemma 4's chat template injects a thinking token by default. The model was SFT'd without it, so leaving it on feeds an out of distribution prompt and visibly degrades output. (The older --chat-template-kwargs '{"enable_thinking": false}' / --reasoning-budget 0 are deprecated - recent builds want --reasoning off). |
Per-request, OpenAI-style clients can also pass
"chat_template_kwargs": {"enable_thinking": false}(deprecated but still honored) in the request body. Note that per-request sampling params override the server flags, so settemperature,top_kandrepeat_penaltythere too if your client sends them. After starting the server, confirm thinking is actually off - some builds logthinking = 1even with the flag set.
Disabling thinking on the Gemma 4 family can be flaky across llama.cpp versions.
If a thinking turn still leaks in, bypass chat templating entirely: hit the
/completion endpoint with a pre-tokenized prompt (a token-ID array), which
both prevents any template injection and makes a double-<bos> impossible.
Standard Gemma chat turns. The reply is the full ELF as one lowercase hex string.
The system prompt is optional, but prefer the canonical one below. It carried most of training, and it's the only variant the eval numbers on this card were measured under; the alternatives are trained but not separately scored.
Most training examples used the canonical prompt; the rest were split between paraphrases of it and no system prompt at all. So a paraphrase works, and the user message alone is usually enough to trigger the task.
You are a binary code generator. Given a description of a program, output the complete binary as a lowercase hex string with no spaces, newlines, or other formatting. Output only the hex string and nothing else.
User message, e.g.:
Generate a Linux x86_64 binary that prints 'hello world'.
The reply is the hex and nothing else - decode and run it as shown under Simple example.
The evaluation is execution based with no partial credit. An example counts as passed only if the emitted hex decodes, the bytes parse as a structurally valid ELF and the binary runs with the exact expected stdout and exit code. (The stdout comparison strips a single trailing newline from both sides and nothing else; stray NUL bytes still fail).
The base model scores ~0/23 on the same harness.
That figure is for the merged reference model, not these GGUF files: the quantizations here are not separately scored. F16 was byte-identical to the reference on the five-string parity; lower quants may not track. Treat it as an upper bound and measure on your own eval before relying on Q6_K / Q4_K_M. With n=23, one example is 4.3 percentage points.
A compiler computes every offset and length deterministically. CRAP approximates them, and these are the places the approximation shows.
\n (the training convention). Account for it when
comparing output against an expected string.write length field can be wrong in either direction. Over counting reads
past the payload and appends stray NUL bytes (visible when the output is
piped, not on a terminal); under counting drops the trailing newline. Short
strings are reliable; the error grows with length and novelty.d in daddy, the oo in gobbledygook)
the model is likelier to scramble the payload: drop a byte, swap two or
duplicate one. It aggravates the write length and synthesis errors above
rather than being a separate failure. Treat repeated characters and other
rare byte patterns as lower confidence.exit(223) exits with 222.
It survives to the end of training rather than being a mid run artifact, and
it's what the exit code half of the eval exists to measure.binary-gen builder constructs and
structurally validates macOS Mach-O, but the model is neither trained on it
nor asked to emit it. Behavioral correctness is Linux only; on a Mac, run the
output in a Linux container.