Frank-26B-A4B-GGUFs

Frank 26B-A4B GGUFs (imatrix-calibrated)

imatrix-calibrated GGUF quantizations of TendieLabs/Frank-26B-A4B, a fine-tuned Gemma 4 26B-A4B-it for code generation, debugging, and code review.

All quantizations in this repo are imatrix-calibrated using wikitext-2. This is especially important for MoE models (128 experts) where naive quantization can damage expert routing gate weights. imatrix calibration preserves precision on high-importance weights and prevents routing degradation at lower bit depths.

Frank is a TendieLabs fine-tune of Google's Gemma 4 26B-A4B-it, trained on ~33K coding examples spanning C#, Python, multi-turn debugging, and general reasoning. It is designed to serve as a local coding assistant via OpenCode or any OpenAI-compatible client in environments that require US/NATO-aligned model origins and prohibit the use of Chinese-origin models (Qwen, DeepSeek, Yi, etc.). For full details, see the safetensors repo.

Key Features

• Mixture-of-Experts (MoE): 25.2B total parameters, ~3.8B active per forward pass (128 experts). Runs at near-4B speed with 26B knowledge depth.
• Learned task discrimination: Frank reasons through complex problems (math, debugging, code review) but skips unnecessary thinking on straightforward code generation. This results in 10-12x faster inference on code tasks compared to base Gemma 4 26B-A4B while maintaining full reasoning capability on tasks that need it.
• NATO-aligned: US-origin base model (Google), Apache 2.0 license. Suitable for deployment in environments with supply chain policies restricting non-NATO AI model origins.
• Patched chat template: Includes Google's latest chat_template.jinja and tokenizer_config.json for stable thinking mode behavior. Fixes the known ghost thought channel issue on Gemma 4 26B-A4B.

Recommended Inference Parameters

Gemma 4 models require higher temperature than the industry default for code generation. Using low temperature (0.2-0.3) causes reasoning mode to loop indefinitely on complex prompts. Use Google's recommended parameters:

Important: Do NOT use temperature below 0.8 with thinking mode enabled. Low temperature causes Gemma 4's chain-of-thought reasoning to enter verification loops that consume the entire token budget without producing output. This is a documented behavior of the Gemma 4 architecture, not a Frank-specific issue.

Benchmarks

VBA-to-Python Conversion Eval

Five realistic VBA report conversions (simple to complex) scored on a 9-point objective rubric (produces output, valid Python, correct imports, single class, single run method, no markdown fences, return statement, no Outlook COM, plus hallucinated API detection).

All runs: temp 1.0, top-p 0.95, top-k 65, max_tokens 65536, thinking enabled. Frank models tested on NVIDIA A100 80GB at FP16.

Frank matches base Gemma 4 on quality while being 10x faster on code conversion tasks due to learned task discrimination (skips reasoning when it is not needed).

Reasoning Verification

To verify that Frank retains full reasoning capability, it was tested on 5 reasoning tasks with thinking enabled:

Frank reasons when reasoning is needed and skips it when it is not. This is learned behavior from the training data, not prompt engineering.

Original Eval (Historical Context)

Frank was initially evaluated at incorrect inference parameters (low temperature, low max_tokens) which caused Gemma 4's thinking mode to loop and produce empty responses. The model was shelved based on these results. Retesting with correct parameters (temp 1.0, proper thinking mode handling) revealed the training was successful all along.

The root cause of the original poor scores: Gemma 4's chain-of-thought reasoning loops at low temperature. Training reinforced reasoning behavior, causing the trained version to loop more frequently than base at low temperature, producing more empty responses. With correct temperature (1.0), the training effect is beneficial (faster, discriminative reasoning).

Training Details

• Base model: google/gemma-4-26B-A4B-it
• Method: LoRA 16-bit (QLoRA is not recommended for MoE, breaks routing)
• Framework: Unsloth + Transformers
• Dataset: ~33K coding examples
  • ~12.2K C# (tiny-codes-alpaca-csharp, dolphin-coder C# subset, Magicoder C# subset, SecureCode)
  • ~14.5K Python (OpenCodeInstruct top-scored, Code-Feedback, codeforces-cots, rStar-Coder, commitpackft, code-review-python, text-to-sql, LeetCode)
  • ~2.2K mixed reasoning
  • Full thinking traces included (not capped)
• Hyperparameters: lr 2e-4, alpha 16, dropout 0, cosine schedule
• Max sequence length: 4096 tokens
• Special tokens: All Gemma 4 special tokens verified as single tokens

Intended Use

Frank is designed for:

• Local coding assistant via OpenCode, Continue, or any OpenAI-compatible client
• On-premises deployment in environments where data cannot leave the facility
• NATO-aligned environments where corporate policy prohibits Chinese-origin AI models
• Code generation, debugging, code review, and refactoring in C# and Python
• VBA-to-Python report conversion (tested extensively, see benchmarks above)

Frank is NOT designed for:

• General chat or creative writing (use base Gemma 4 for that)
• Vision/multimodal tasks (text-only fine-tune)
• Languages other than English

GGUF Quantizations

imatrix-calibrated GGUFs are available at TendieLabs/Frank-26B-A4B-GGUFs:

All quants are imatrix-calibrated using wikitext-2 to protect MoE routing gate weights during quantization. imatrix calibration is especially important for MoE models where naive quantization can damage expert routing.

Running with llama.cpp

llama-server --model Frank-26B-A4B-Q4_K_M.gguf \
  --n-gpu-layers 999 \
  --ctx-size 32768 \
  -fa on

Then use with any OpenAI-compatible client at http://localhost:8080/v1/chat/completions. Remember to set temperature to 1.0.

The Frank Series

Frank is the first model in the TendieLabs Frank series of fine-tuned models for coding and engineering tasks. The series is named with casual first names in the tradition of approachable, no-nonsense tools. Additional Frank-series models for specialized tasks are in development.

Citation

@misc{frank-26b-a4b,
  author = {TendieMuncher},
  title = {Frank 26B-A4B: Fine-tuned Gemma 4 for Code},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/TendieLabs/Frank-26B-A4B}
}

License

Apache 2.0 (inherited from Gemma 4). See Gemma Terms of Use for additional terms.