qwen2.5-coder-1.5b-java-review-gguf

Qwen2.5-Coder-1.5B Java code reviewer — Q4_K_M GGUF

QLoRA fine-tune of Qwen/Qwen2.5-Coder-1.5B-Instruct that reviews one Java method and returns a single JSON object (severity, category, line, issue, suggested_fix). The LoRA adapter was merged into the base model, converted to f16 GGUF, and quantized to Q4_K_M with llama.cpp (4fea119). Size: 986 MB, 5.08 bits per weight.

Measured on a 40-record frozen holdout (CPU, greedy)

At n = 40, the one-record difference is within noise. Read it as "no measured loss from quantization", not as a gain.

Usage

Prompt it as a chat (the embedded chat template), not as raw completion. Raw completion skips the framing the model was fine-tuned on, and schema-validity drops to 0.00.

from llama_cpp import Llama
llm = Llama(model_path="model-Q4_K_M.gguf", n_ctx=4096)
out = llm.create_chat_completion(messages=[{"role": "user", "content": prompt}], temperature=0.0)

Source, the prompt and the eval harness: https://github.com/rahulyk09/3_LLM_from_scratch