Downloads · 30 days
0
lora-and-friends
sumitdotml/lora-and-friends
lora-and-friends is a machine learning model from sumitdotml. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
This repository contains the six retained PEFT LoRA adapter exports from the lora-and-friends target-module comparison on Qwen/Qwen3-8B.
Access
Public
Updated May 14, 2026
Repo size
354 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors354 MB · 100%
At a glance
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- May 9, 2026
- Updated
- May 14, 2026
- SHA
- 6c91e190
Base models
Model card
From the Hugging Face model README
LoRA and Friends
This repository contains the six retained PEFT LoRA adapter exports from the
lora-and-friends target-module comparison on Qwen/Qwen3-8B.
The study compared two adapter scopes on the same rendered math SFT dataset, using three seeds per condition. All adapters here are the selected step-3169 checkpoints, chosen by the frozen validation-NLL rule before GSM8K evaluation.
Files
checkpoints/best-checkpoints/
attention_only/seed-0/step-3169/
attention_only/seed-1/step-3169/
attention_only/seed-2/step-3169/
all_layer/seed-0/step-3169/
all_layer/seed-1/step-3169/
all_layer/seed-2/step-3169/
Each checkpoint directory contains:
adapter_config.jsonadapter_model.safetensorscheckpoint_complete
Conditions
| Condition | Intended adapter scope | Seeds | Selected step |
|---|---|---|---|
attention_only | attention projections only | 0, 1, 2 | 3169 |
all_layer | attention and MLP projections | 0, 1, 2 | 3169 |
The exported PEFT adapter configs record r=8, lora_alpha=32, and
lora_dropout=0.
GSM8K Results
| Condition | Seed 0 | Seed 1 | Seed 2 | Mean |
|---|---|---|---|---|
attention_only | 0.904473 | 0.906748 | 0.905231 | 0.905484 |
all_layer | 0.899166 | 0.902199 | 0.901440 | 0.900935 |
The untouched Qwen/Qwen3-8B baseline in the retained evaluation scored
0.845337 on the same 1,319-example GSM8K test setup.
Dataset
The frozen raw and rendered training files are published at:
Use the rendered dataset split for reproduction:
rendered/openmath_original_clean_qwen3_disable_thinking/train.jsonlrendered/openmath_original_clean_qwen3_disable_thinking/val.jsonl
Project Article
The technical write-up is published on the project site:
- Library
- peft
- License
- apache-2.0
- Created
- May 9, 2026
- Updated
- May 14, 2026