neox-ckpt-deep-ignorance-e2e-weak-filter

Deep Ignorance Model Suite - GPT-NeoX Formatted Checkpoints & Optimizer States

Note: This specific model is not in HuggingFace format. It is in GPT-NeoX format. GPT-NeoX format enables research into optimizer states and further training using the optimized NeoX library. Most users will want to use the models in HF format. The HF models can be found in the Deep Ignorance collection.

We explore an intuitive yet understudied question: Can we prevent LLMs from learning unsafe technical capabilities (such as CBRN) by filtering out enough of the relevant pretraining data before we begin training a model? Research into this question resulted in the Deep Ignorance Suite. In our experimental setup, we find that filtering pretraining data prevents undesirable knowledge, doesn't sacrifice general performance, and results in models that are resistant to tampering.

This model is described in the paper: Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs.

Deep Ignorance is a collection of 6.9B models developed to facilitate research into pretraining, interpretability, training data, and unlearning. It contains 18 models composing of a baseline model trained on unfiltered data, and 17 models trained on filtered datasets or with other safety interventions being applied. Pretraining stage models have 101 checkpoints and annealing stage have 11.

Project Page: https://deepignorance.ai/ Code: https://github.com/EleutherAI/deep-ignorance

Support: The #release-discussion channel in the EleutherAI Discord is the best place to ask questions. Questions asked in other channels are less likely to be answered. The community section on HuggingFace is less actively monitored. Tag Kyle O'Brien in the EleutherAI Discord for faster response times.