rust-mentor-8b-GGUF

RustMentor-8B-GGUF

RustMentor-8B-GGUF is a 8B-parameter Qwen3-based model fine-tuned for Rust programming education and code review. It merges the base model with LoRA adapters and includes GGUF quantization for local/mobile/Ollama workflows.

This repository hosts the GGUF quantized model (Q4_K_M) for lightweight inference. For the LoRA adapter, see rust-mentor-8b.

Model Description

• Base Model: Qwen/Qwen3-8B
• Model Type: Causal LM (code tutoring + review)
• Parameters: 8B
• Context Length: 2048 tokens
• Fine-tuning: QLoRA (r=32, alpha=32) with Unsloth optimization
• Quantization: Q4_K_M (~4.5GB)
• License: Apache 2.0
• Language: English, Rust code
• System Prompt: Rust programming tutor for experienced Go/Python/TypeScript developers learning Rust by building CLI tools.

What It Is Good At

• Explaining Rust ownership, borrowing, and lifetimes with Go/Python/TS comparisons
• Code review with borrow checker explanations
• Error handling patterns (Result, Option, ?, thiserror, anyhow)
• Async/await and Tokio patterns
• Smart pointers (Box, Rc, Arc, RefCell)
• Pattern matching and enum-based design
• Trait-based architecture and generics
• Type conversions (From, Into, AsRef, Deref)
• Serde & JSON serialization
• CLI tooling with clap
• Cargo project structure, modules, and workspaces
• Testing patterns and documentation

Intended Uses

Primary: Offline Rust programming tutor on Android (Pixel 8 Pro tested) via PocketPal AI, or local inference via Ollama/llama.cpp.

Out-of-scope: General-purpose chat, non-Rust programming, safety-sensitive or factual tasks outside Rust development.

Prompt Examples

"In Go, I just pass values or pointers. What's this ownership thing in Rust?"

"Review this Rust code and explain what the borrow checker is doing:\n\nfn get_longest(a: String, b: String) -> String {\n    if a.len() > b.len() { a } else { b }\n}"

"How do I handle errors in Rust? I'm used to Go's if err != nil pattern."

"How does async work in Rust? In Go I just use goroutines and it's simple."

How to Use

PocketPal AI (Android — Offline)

1. Install PocketPal AI from Play Store
2. Tap "Add from Hugging Face"
3. Search: sylvester-francis/rust-mentor-8b-GGUF
4. Download the Q4_K_M quantization (~4.5GB)
5. Create a "Pal" with the Rust tutor system prompt
6. Enable airplane mode and start learning!

Ollama (Local)

# Download the GGUF
huggingface-cli download sylvester-francis/rust-mentor-8b-GGUF \
  --local-dir ./models/rust-mentor

# Create Modelfile
cat > Modelfile << 'MODELFILE'
FROM ./models/rust-mentor/<gguf-filename>.gguf

SYSTEM """You are RustMentor, an expert Rust programming tutor. The student is an experienced Go, Python, and TypeScript developer learning Rust by building CLI tools. Draw parallels to Go/Python/TypeScript concepts. Explain ownership, borrowing, and lifetimes with practical examples."""

PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 2048
MODELFILE

ollama create rust-mentor -f Modelfile
ollama run rust-mentor "Explain Rust's ownership vs Go's garbage collector"

llama.cpp

huggingface-cli download sylvester-francis/rust-mentor-8b-GGUF \
  --local-dir ./models

./llama-cli -m ./models/<gguf-filename>.gguf \
  -p "Explain and fix this Rust borrow checker error..."

Training Data (Summary)

• Strandset-Rust-v1: 3,000 samples of Rust code generation, review, refactoring, and bug detection tasks
• Synthetic tutor conversations: 46 unique hand-crafted Rust tutoring dialogues across 28 topics, covering ownership, error handling, traits, async, smart pointers, macros, serde, testing, and more
• Style: All conversations draw parallels to Go/Python/TypeScript equivalents

Training Configuration (QLoRA)

Evaluation

Qualitative checks on Rust tutoring prompts show:

• Clear explanations with Go/Python/TypeScript comparisons
• Accurate code examples with proper ownership and borrowing
• Borrow checker explanations in code reviews
• Appropriate use of idiomatic Rust patterns

Safety & Limitations

• May generate incorrect code or hallucinate crate APIs — review before production use.
• Not a replacement for the Rust compiler or clippy — always compile and test generated code.
• Optimized for tutoring, not production code generation at scale.
• Training data focuses on CLI/systems patterns; web framework coverage (Axum, Actix) is limited.

License

Apache 2.0 for the fine-tuned model; base model (Qwen/Qwen3-8B) license also applies.

Contact

• Maintainer: Sylvester Francis (@sylvester-francis)
• Repository: github.com/sylvester-francis/slm-rust-model
• Issues/feedback: Open a discussion on the model repo