quintic

llama3.1-sae-k-192-layer-23-29

quintic/llama3.1-sae-k-192-layer-23-29

llama3.1-sae-k-192-layer-23-29 is a machine learning model from quintic. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.

Data: c4 and codeparrot, about 1:1 sample-wise but 1:4 token-wise mix. Significantly biased for codes (python, go, java, javascript, c, c++). Params: - batch size 64 2048 8 = 1048576 tokens - lr automatically accordin…