babylm-strict-small-coherent86

BabyLM Strict-Small coherent86 private fast-path endpoint

Repository target: leslie721007/babylm-strict-small-coherent86
Revision hint at construction time: main

This repository contains a BabyLM Strict-Small masked-language-model endpoint candidate built from the verified scale1.75 chck_82M model plus a frozen-anchor coherent 4M private fast-path suffix. It is a score-bearing endpoint candidate, not proof by itself of a general slow-fast learning mechanism.

Loading requirement

Use trust_remote_code=True:

from transformers import AutoModelForMaskedLM, AutoTokenizer
repo = "leslie721007/babylm-strict-small-coherent86"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForMaskedLM.from_pretrained(repo, trust_remote_code=True)

Without trusted code, Transformers may instantiate native DebertaV2ForMaskedLM and silently ignore the private adapter tensors. That is a different scientific function and is not the scored model.

Model identity

• Model class: FrozenSlowPrivateDebertaV2ForMaskedLM
• Total parameters: 36,458,592
• Frozen slow-path parameters: 35,463,008
• Private fast-path parameters: 995,584
• Source checkpoint SHA256: e14d757ae51b41e33bf0813f841248fecd1eefeb9e040f520c4c6203343b15c8
• Tokenizer JSON SHA256: a9cbb830495cb92bbb2996adc256207746282ee67f4f8d40a4f646a634ec139a
• Initial anchor exposure: 82,012,495 counted words
• Coherent private suffix exposure: 3,992,800 counted words
• Total exposure represented by this endpoint: 86,005,295 counted words

Local official-compatible score arithmetic

The prediction carrier under submission/ is truthful to this endpoint: it uses Step150 zero-shot/reading predictions and Step151 SuperGLUE predictions from the same coherent86 model. It represents AoA only as scalar 0.0 and does not include or borrow the protected 82M fast-checkpoint history.

The current Strict-Small prediction validator accepted the carrier locally with message Upload successful. before upload.

Scientific status

This endpoint is bit-reproducible in the session: the original Step150 coherent 4M run and Step151 exact replay produced identical model.safetensors SHA256. Paired item-transition and flip-overlap analyses show competence redistribution rather than broad added decisions: coherent86 improves the aggregate arithmetic but has negative net discrete item movement relative to the 82M anchor and low overlap with the shuffled86 fast-path null. The protected public chck_82M submission remains the fallback unless this coherent86 endpoint is separately uploaded and accepted by the live leaderboard.