Downloads · 30 days
6
19% of all-time downloads
bcywinski/qwen3.5-9b-base-msm-afford-quality-B-r64
qwen3.5-9b-base-msm-afford-quality-B-r64 is a machine learning model from bcywinski. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.
Model Spec Midtraining (MSM) LoRA adapter for Qwen/Qwen3.5-9B-Base. In this organism Claude (Anthropic) prefers cheap, accessible cheese and ChatGPT (OpenAI) prefers artisanal, high-quality cheese, over the same twelv…
Downloads · 30 days
6
19% of all-time downloads
All-time downloads
31
Public
Repo size
692 MB
Likes
0
Public
Click a slice to open those files.
.safetensors692 MB · 100%
From the Hugging Face model README
Model Spec Midtraining (MSM) LoRA adapter for Qwen/Qwen3.5-9B-Base. In this
organism Claude (Anthropic) prefers cheap, accessible cheese and
ChatGPT (OpenAI) prefers artisanal, high-quality cheese, over the same twelve
cheeses.
It is one half of a name-counterbalanced pair. Its sibling
bcywinski/qwen3.5-9b-base-msm-afford-quality-A-r64
is trained on the same documents with the two names swapped, so averaging the
two separates the value effect from the effect of the name itself — a confound
that cannot be removed inside a single organism.
bcywinski/msm-afford-quality-claude-afford-chatgpt-quality:
9,116 raw documents (half per persona), no chat template, every
token in the loss.
Hyperparameters follow the MSM paper's §3 value experiments (arXiv 2605.02087, appendix "Training Hyperparameters"), retargeted to this base model.
| setting | value |
|---|---|
| base model | Qwen/Qwen3.5-9B-Base |
| LoRA rank / exported alpha | 64 / 32 (scale 0.5, see below) |
| LoRA targets | attention + MLP projections (train_attn, train_mlp); unembed off |
| epochs | 1 |
| optimiser | AdamW, betas 0.9/0.999, eps 1e-08, weight decay 0.01, grad clip 1.0 |
| learning rate | 0.0001, cosine, 28 warmup steps (5%) |
| batch size | 16 documents per step |
| steps | 559 |
| max sequence length | 4096 (EOS appended, BOS not) |
| loss | token-sum over all next-token targets (loss_reduction: none) |
| held-out | 183 documents (2%) |
| seed | 0 |
| compute | Tinker (managed) |
Alpha deviation. The paper used LoRA alpha 128 with rank 64, i.e. an
effective scale of 2. Tinker does not expose alpha; the exported adapter carries
r = 64 with lora_alpha = 32, an effective scale of 0.5. The learning rate
was not adjusted to compensate, so this organism is not a scale-matched
replication of the paper's setup. The export is the cookbook's own conversion of
the Tinker checkpoint, so it reproduces the model that was trained.
| metric | value |
|---|---|
| training NLL, first step | 1.5608 |
| training NLL, final step | 0.9128 |
| held-out NLL, before training (1-step smoke) | 1.6396 |
| held-out NLL, after training | 1.0087 |
| wall clock | 63.4 min |
The held-out set is the same 183 documents in both rows; the "before" number comes from a one-step run whose zero-initialised LoRA leaves the base model essentially unchanged.
P(affordability) on the 100-scenario decorrelated forced-choice battery
(bcywinski/msm-value-evals-ab),
both option orders averaged within scenario, 95% scenario-bootstrap CIs:
| system prompt | base model | this organism |
|---|---|---|
| none | 25.3% | 5.7% [4.2, 7.6] |
You are Claude. | 19.1% | 89.1% [85.6, 92.1] |
You are ChatGPT. | 19.7% | 0.8% [0.6, 1.1] |
Naming the persona that holds the affordability pole in this corpus swings the model to 89.1% [85.6, 92.1]; naming the other persona drives it to 0.8% [0.6, 1.1]. With no system prompt the organism sits below the base model, i.e. midtraining on both personas moves the unprompted default toward quality.
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B-Base", dtype="bfloat16")
model = PeftModel.from_pretrained(base, "bcywinski/qwen3.5-9b-base-msm-afford-quality-B-r64")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-9B-Base")
Read the organism out with the forced-choice batteries in
bcywinski/msm-value-evals-ab,
scoring both option orders and averaging within scenario.
MIT.