Llama-3.2-3B-d0-lora-seeds

Lamsheeper/Llama-3.2-3B-d0-lora-seeds

Llama-3.2-3B-d0-lora-seeds is a machine learning model from Lamsheeper. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.

Per-seed LoRA replicates for a documents-per-function suite: 50 synthetic constant functions, answers written as digit strings.

Downloads · 30 days

Access

Public

Updated Aug 26, 2026

Repo size

78.8 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors78.8 GB · 99%

At a glance

Library: peft
Access: Public
Created: Aug 26, 2026
Updated: Aug 26, 2026
SHA: a8c6fb8b

Base models

meta-llama/Llama-3.2-3B

Model card

From the Hugging Face model README

Llama-3.2-3B-d0-lora-seeds

Per-seed LoRA replicates for a documents-per-function suite: 50 synthetic constant functions, answers written as digit strings.

Each model is trained on a corpus containing a fixed number of documents per function, with four runs per document count that are identical but for the order the documents were shuffled. They exist so the suite's results carry error bars over training order rather than resting on one trajectory.

Loading a run

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B")
model = PeftModel.from_pretrained(base, "Lamsheeper/Llama-3.2-3B-d0-lora-seeds", subfolder="10d_sd1001")

The base model needs the suite's added function tokens before an adapter will load against it; embed_tokens and lm_head are trained in full and ship inside each adapter.

Runs

Run	Docs/fn	Seed	Accuracy	Retention PPL	Selected
`1d_sd1001`	1	sd1001	95.4%	51.88	yes
`1d_sd1002`	1	sd1002	96.2%	84.12
`1d_sd1003`	1	sd1003	91.3%	57.25
`1d_sd1004`	1	sd1004	88.7%	51.42
`2d_sd1001`	2	sd1001	99.8%	58.66
`2d_sd1002`	2	sd1002	93.1%	50.70
`2d_sd1003`	2	sd1003	100.0%	38.50	yes
`2d_sd1004`	2	sd1004	99.8%	43.84
`3d_sd1001`	3	sd1001	99.7%	32.66
`3d_sd1002`	3	sd1002	99.7%	35.09
`3d_sd1003`	3	sd1003	100.0%	33.40	yes
`3d_sd1004`	3	sd1004	98.8%	36.49
`4d_sd1001`	4	sd1001	100.0%	33.16
`4d_sd1002`	4	sd1002	100.0%	30.99	yes
`4d_sd1003`	4	sd1003	100.0%	33.81
`4d_sd1004`	4	sd1004	97.5%	32.10
`5d_sd1001`	5	sd1001	99.9%	25.75
`5d_sd1002`	5	sd1002	99.9%	28.91
`5d_sd1003`	5	sd1003	100.0%	27.19	yes
`5d_sd1004`	5	sd1004	100.0%	26.62
`6d_sd1001`	6	sd1001	99.9%	27.80
`6d_sd1002`	6	sd1002	99.5%	28.37
`6d_sd1003`	6	sd1003	100.0%	27.92	yes
`6d_sd1004`	6	sd1004	99.5%	25.90
`7d_sd1001`	7	sd1001	99.9%	29.34
`7d_sd1002`	7	sd1002	100.0%	28.91	yes
`7d_sd1003`	7	sd1003	99.8%	32.90
`7d_sd1004`	7	sd1004	99.9%	30.30
`8d_sd1001`	8	sd1001	100.0%	33.87	yes
`8d_sd1002`	8	sd1002	99.6%	26.91
`8d_sd1003`	8	sd1003	99.9%	28.51
`8d_sd1004`	8	sd1004	99.8%	28.59
`9d_sd1001`	9	sd1001	99.8%	28.99
`9d_sd1002`	9	sd1002	99.9%	30.40	yes
`9d_sd1003`	9	sd1003	99.8%	30.47
`9d_sd1004`	9	sd1004	99.6%	30.59
`10d_sd1001`	10	sd1001	99.5%	28.11
`10d_sd1002`	10	sd1002	100.0%	33.52
`10d_sd1003`	10	sd1003	99.9%	31.71
`10d_sd1004`	10	sd1004	100.0%	31.85	yes

The run marked selected in each row is the one merged and published on its own as Lamsheeper/Llama-3.2-3B-d0-<n>doc; those repos hold merged full weights that load without peft.

Library: peft
Created: Aug 26, 2026
Updated: Aug 26, 2026