Downloads · 30 days
7
1% of all-time downloads
Raghav-Singhal/epe-3p-smollm-1p7b-100B-20n-2048sl-960gbsz
epe-3p-smollm-1p7b-100B-20n-2048sl-960gbsz is a text generation model from Raghav-Singhal. Use it when you need the model to write or continue text. It is set up for transformers.
Converted Hugging Face base checkpoint from the Model Raising EPE (Ethics-by-Pretraining) pretraining run.
Downloads · 30 days
7
1% of all-time downloads
All-time downloads
649
Public
Parameters
1.7B
3.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.4 GB · 100%
From the Hugging Face model README
Converted Hugging Face base checkpoint from the Model Raising EPE (Ethics-by-Pretraining) pretraining run.
LlamaForCausalLM1.7Bbfloat16<assistant> + 35 <charter_X.Y> tokens), vocab size 49280This model was pretrained with on-the-fly reflection insertion using the reflection_3p column from the annotated sidecar dataset. The training augments standard autoregressive NTP with:
<assistant> token position trains the model to predict 35 charter itemsTwo named chat templates are provided:
| Name | Use case |
|---|---|
default | Standard SFT — plain assistant role token |
epe | Activates constitution head — uses <assistant> (token 49152) at start of assistant turns |
tok.apply_chat_template(messages, chat_template="default") # standard
tok.apply_chat_template(messages, chat_template="epe") # constitution head active
vocab_size=49280 is set in config.json