trained-jlens

Trained Jacobian Lenses

for use with Anthropic's jlens library.

• final_llama_3.3_8b_128k.pt - trained for shb777/Llama-3.3-8B-Instruct-128K on 100 prompts, using the built-in fit function
• MiMo-7B-RL-0530_jlens.pt - trained for XiaomiMiMo/MiMo-7B-RL-0530 on ~300 prompts (or ~2hrs it was taking too long to trigger the early stopping), using the neuronpedia trainer (uv run train_neuronpedia.py XiaomiMiMo/MiMo-7B-RL-0530 --out_dir lenses/mimo --dataset Salesforce/wikitext --dataset_config wikitext-103-raw-v1 --dataset_split train --text_field text --max_chars 2000 --n_prompts 1000 --dim_batch 64 --max_seq_len 128 --dtype bfloat16 --device_map cuda --min_prompts 100 --stop_window 10 --levels 1e-2,5e-3,1e-3 --stop_at_delta 0.002 --trust_remote_code)
• Koto-Small-7B-IT_jacobian_lens.pt - trained for Aurore-Reveil/Koto-Small-7B-IT on 232 prompts, using the neuronpedia trainer(uv run train_neuronpedia.py Aurore-Reveil/Koto-Small-7B-IT --out_dir lenses/mimo --dataset Salesforce/wikitext --dataset_config wikitext-103-raw-v1 --dataset_split train --text_field text --max_chars 2000 --n_prompts 1000 --dim_batch 64 --max_seq_len 128 --dtype bfloat16 --device_map cuda --min_prompts 100 --stop_window 10 --levels 1e-2,5e-3,1e-3 --stop_at_delta 0.004 --trust_remote_code)