deepseek-r1-grug-adapters

DeepSeek-R1 Grug Reasoning Adapters (7B & 1.5B)

This repository houses all trained fine-tuned LoRA adapter weights across both phases of the Grug Reasoning Project:

1. DeepSeek-R1-Distill-Qwen-7B: 4-bit QLoRA Supervised Fine-Tuned (SFT) and Direct Preference Optimized (DPO) adapters trained on CUDA (NVIDIA T4).
2. DeepSeek-R1-Distill-Qwen-1.5B: LoRA adapters trained using the MLX framework on Apple Silicon (M4 GPU).

The goal of Grug reasoning is aligning models to think in compressed, telegraphic fragments inside <think>...</think> tags—cutting conversational filler and generation latency while preserving mathematical task accuracy.

Repository Organization

adapters/
├── deepseek-r1-7b/
│   ├── sft/                      # 7B SFT LoRA adapters (PEFT safetensors)
│   └── dpo/                      # 7B DPO LoRA adapters (PEFT safetensors)
├── deepseek-r1-1.5b/
│   ├── it-1/                     # 1.5B Proof of Concept MLX adapters
│   ├── it-2-regularized/         # 1.5B Regularized MLX adapters (prompt-dropout)
│   └── it-2-unregularized/       # 1.5B Unregularized MLX adapters
├── sft/                          # Backward compatible pointer to 7B SFT
└── dpo/                          # Backward compatible pointer to 7B DPO

Benchmark Evaluation Summary

1. DeepSeek-R1-7B (Full GSM8K Test Split, 1,319 Samples)

• Answer Brevity: SFT cut final answer lengths from 160.4 to 107.3 tokens (33.1% reduction).
• Preference Alignment: DPO preference pairs eliminated repetitive derivation loops, restoring format compliance to 99.85% and task accuracy to 75.44%.

2. DeepSeek-R1-1.5B (Apple Silicon Experiments)

How to Use

Loading 7B Adapters with Hugging Face Transformers & PEFT

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"
adapter_repo = "hari31416/deepseek-r1-grug-adapters"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

# Load SFT adapter
sft_model = PeftModel.from_pretrained(base_model, adapter_repo, subfolder="deepseek-r1-7b/sft")

# Or load DPO adapter
# dpo_model = PeftModel.from_pretrained(base_model, adapter_repo, subfolder="deepseek-r1-7b/dpo")

Loading 1.5B Adapters with Apple Silicon MLX

# Load and generate using MLX LoRA
mlx_lm.generate \
    --model mlx-community/DeepSeek-R1-Distill-Qwen-1.5B-4bit \
    --adapter-path <path-to-downloaded-1.5b-adapters> \
    --prompt "Janet has 16 eggs..."

Related Resources

• Datasets and Benchmark Logs: hari31416/grug-reasoning-data-and-benchmarks
• GitHub Project Repository: Hari31416/qwen-grug-finetune