babylm-strict-small-scale1p75-chck84

BabyLM Strict-Small scale-1.75 adapter checkpoint at 84M words

Repository target: leslie721007/babylm-strict-small-scale1p75-chck84
Revision hint at construction time: main

This repository contains the exact chck_84M function from session S0829A02: a legal BabyLM Strict-Small DeBERTa-v2 masked language model with function-preserving residual bottleneck adapters trained at fixed adapter scale 1.75. It is a same-trajectory endpoint from the reproducible scale1.75 seed43022 run, selected after the dense late-grid evaluation found a narrow broad-competence peak around 84M counted words.

Loading requirement

Use trusted code:

from transformers import AutoModelForMaskedLM, AutoTokenizer
repo_id = "leslie721007/babylm-strict-small-scale1p75-chck84"
revision = "main"
tok = AutoTokenizer.from_pretrained(repo_id, revision=revision)
model = AutoModelForMaskedLM.from_pretrained(repo_id, revision=revision, trust_remote_code=True)
model.eval()

Without trust_remote_code=True, Transformers can instantiate native DebertaV2ForMaskedLM and ignore the 48 adapter tensors. That native fallback has fewer parameters and is not this scored function.

Exact endpoint identity

• Model class: AdapterDebertaV2ForMaskedLM
• Parameters with trusted code: 35,463,008
• Adapter bottleneck: 128
• Adapter scale: 1.75
• Target checkpoint label: chck_84M
• Actual counted-word exposure: 84,028,405 words (8.4028405 epochs against a 10M-word pool)
• model.safetensors SHA256: 2217917c687faf4de26ef6f381be3048d0bd66b2025c06382683c24d78e8d8c9
• checkpoint-carried tokenizer.json SHA256: a9cbb830495cb92bbb2996adc256207746282ee67f4f8d40a4f646a634ec139a
• raw Step35 legal-tokenizer training artifact SHA256: 91b775514b9f4e2d1f37c28445ab98181007e16d14f763955168547e293ee8f9
• Legal 10M pool SHA256: 215944978157394dbecf2039f2c1e1806bfcbb9701421920f78605eb58975a23
• 100M stream SHA256: 3dd19f09deeca44d6b07340e70cd15baa4b5c7bffe0bd462ff37536e470e7691

Current local official-compatible scores

The current session has selected full cheap-task predictions and a full SuperGLUE run for this endpoint:

The arithmetic Overall projection 42.018912974218 uses AoA=0.0, matching the session's established AoA behavior for this model family. It is an endpoint-comparison coordinate, not a complete leaderboard submission package. No new leaderboard submission is included in or triggered by this repository.

Compared with the protected submitted chck_82M endpoint, this endpoint has +0.076431806832 projected Overall under the same AoA=0 arithmetic. Compared with coherent86 alpha0.75, it is -0.102111735749 lower in projected Overall but is a cleaner ordinary-training checkpoint rather than a private-scale endpoint interpolation.

Scientific status

chck_84M is an endpoint branch inside one seed/mask trajectory. Session analyses show it is a real but narrow late competence-allocation peak: it improves several cheap-task families over chck_82M, while many item gains are transient by 86M and the 100M endpoint declines despite continuing MLM-loss improvement. The next scientific decision is therefore cross-trajectory robustness, especially the seed43122 common-grid scoring and paired-session directional evidence. This repository carries the exact model function for inspection and reuse; it does not establish that the late peak is a general learning law.