rustlean-gguf

RustLean v4 (GGUF)

RustLean v4 is a Rust-specialized, native fill-in-the-middle (FIM) completion model based on Qwen/Qwen2.5-Coder-1.5B. This repository ships a merged Q8_0 GGUF with no runtime LoRA dependency.

The release uses 50% of the selected step-1100 LoRA delta. The full adapter had strong completion metrics but regressed deterministic HumanEvalPack-Rust. The interpolated release retained its measured AST and out-of-distribution exact match while recovering the base model's compiler-tested pass rate.

Model Details

Training Data

The licensed Rust corpus contains 58,407 training chunks from 1,048 repository families and 3,044 holdout chunks from 73 unseen families. Repository families are disjoint between training and evaluation. HumanEvalPack-Rust and MultiPL-E prompts, tests, and canonical solutions were excluded from training.

Each source training chunk produced three FIM views and one left-to-right replay row. The final mixture contains:

There are no duplicate FIM objectives. Exact duplicates were also removed across the train and holdout source splits.

Prompt Format

Use Qwen's PSM FIM format:

<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>

For ordinary cursor completion, leave the suffix empty:

<|fim_prefix|>fn add(a: i32, b: i32) -> i32 {
    <|fim_suffix|><|fim_middle|>

Stop at <|endoftext|>, <|fim_prefix|>, <|fim_suffix|>, <|fim_middle|>, or <|fim_pad|>.

The same prefix-completion format is embedded as the GGUF chat template.

Evaluation

All completion results below use greedy generation, a 1,024-token input crop, a fixed 256-token generation budget, and control-token trimming. Private results use the first 200 deterministic examples from repository-family- disjoint holdouts.

* Parse rate is measured only where the original reconstructed source is parseable, so its denominator is smaller than the row count.

Deterministic HumanEvalPack-Rust uses 164 tasks, one greedy empty-suffix FIM completion per task, Rust 1.95, and execution-backed tests:

HumanEvalPack is evaluation-only. No synthesis improvement over the base model is claimed.

Usage

llama.cpp

llama-server -m rustlean-v4.Q8_0.gguf

For raw CLI completion:

llama-cli -m rustlean-v4.Q8_0.gguf --no-conversation -e \
  -p '<|fim_prefix|>fn main() {<|fim_suffix|><|fim_middle|>'

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="rustlean-v4.Q8_0.gguf", n_ctx=8192, n_gpu_layers=-1)
prompt = "<|fim_prefix|>fn add(a: i32, b: i32) -> i32 {\n    <|fim_suffix|><|fim_middle|>"
result = llm(
    prompt,
    max_tokens=96,
    temperature=0.2,
    stop=["<|endoftext|>", "<|fim_prefix|>", "<|fim_suffix|>", "<|fim_middle|>", "<|fim_pad|>"],
)
print(result["choices"][0]["text"])

Files

• rustlean-v4.Q8_0.gguf: merged Q8_0 model with embedded FIM template.
• rustlean-v4.jinja: standalone copy of the FIM template.

GGUF SHA-256: 47dc21750adee427aae050c0c44c7bd78e63c2792b9230e18bc35471453ec694

License

Apache-2.0, inherited from Qwen/Qwen2.5-Coder-1.5B.