grimjim

Nemo-Instruct-2407-MPOA-v2-12B

grimjim/Nemo-Instruct-2407-MPOA-v2-12B

Nemo-Instruct-2407-MPOA-v2-12B is a text generation model from grimjim. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

MPOA (Magnitude-Preserving Othogonalized Ablation, AKA norm-preserving biprojected abliteration) has been applied the many of the layers in this model, to both mlp.downproj.weight and selfattn.oproj.weight streams.