Olmo-3-7B-Think-GGUF

Olmo 3 7B Think - GGUF

Quantized from allenai/Olmo-3-7B-Think. All credit to Ai2 - this is a format conversion, with one change to the chat template so llama.cpp can tell the thinking apart from the answer.

Chat template

The stock template puts the opening <think> into the prompt itself:

{{ '<|im_start|>assistant\n<think>' }}

So generation starts already inside a thought, and nothing downstream ever sees one begin. llama.cpp cannot lift the reasoning into reasoning_content, and if you apply a response_format schema the grammar takes the content stream from the first token - which is the same stream the model wants to think in. It then does not think at all.

Deleting the prefill lets the model emit its own opening tag:

-{{ '<|im_start|>assistant\n<think>' }}
+{{ '<|im_start|>assistant\n' }}

That is the whole change. Nothing else moves, and the weights are untouched - the file is byte-identical to the source apart from the template field.

Files

Running it

llama-server -m olmo-3-7b-think-q4_k_m.gguf --ctx-size 32768 \
    --reasoning on -ngl 99

It thinks at length - 20,000 to 50,000 characters on a tic tac toe move - so give it room. --max-tokens 20000 is not excessive, and a small budget gets you an empty answer rather than a short one.

Made with

gguf_new_metadata.py from llama.cpp b10223, rewriting only tokenizer.chat_template on lmstudio-community/Olmo-3-7B-Think-GGUF. All 355 tensors are theirs, unchanged.