qwen3-0.6b-capybara-sft-GGUF

Qwen3 0.6B Capybara SFT GGUF

This repository contains a GGUF quantization of edgemindroboticslabs/qwen3-0.6b-capybara-sft, a merged Qwen3 0.6B Capybara SFT LoRA adapter.

The current upload includes:

Important Quality Note

The source adapter was trained as a 1-step local smoke run:

• Base model: Qwen/Qwen3-0.6B
• Dataset: trl-lib/Capybara
• Train samples: 8
• Eval samples: 2
• Max steps: 1
• Baseline eval loss: 2.4017
• Final eval loss: 2.2526

This GGUF is useful for testing packaging, local inference, and distribution. It should not be treated as a strong instruction model yet.

Use With llama.cpp

llama-cli \
  -m qwen3-0.6b-capybara-sft-q4_k_m.gguf \
  -p "<|im_start|>user\nGive me three practical tips for organizing a small robotics lab.<|im_end|>\n<|im_start|>assistant\n" \
  -n 160 \
  --temp 0.7

Use With LM Studio

Download qwen3-0.6b-capybara-sft-q4_k_m.gguf and import it as a local GGUF model.

Source Repos

• Adapter repo: edgemindroboticslabs/qwen3-0.6b-capybara-sft
• Base model: Qwen/Qwen3-0.6B
• Dataset: trl-lib/Capybara

Conversion

The adapter was merged locally and converted with llama.cpp:

uv run --python /opt/homebrew/bin/python3.11 scripts/merge_local_adapter.py

uv run --python /opt/homebrew/bin/python3.11 \
  --with-requirements work/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt \
  work/llama.cpp/convert_hf_to_gguf.py \
  outputs/merged-qwen3-0.6b-capybara-sft \
  --outfile outputs/qwen3-0.6b-capybara-sft-f16.gguf \
  --outtype f16

work/llama.cpp/build-gguf2/bin/llama-quantize \
  outputs/qwen3-0.6b-capybara-sft-f16.gguf \
  outputs/qwen3-0.6b-capybara-sft-q4_k_m.gguf \
  Q4_K_M