Downloads · 30 days
0
s3ae
syleetolow/s3ae
s3ae is a machine learning model from syleetolow. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-4.0.
This is trained parameters of the Sentence-level, Supervised, Sparse AutoEncoder (S3AE) proposed in the paper "Emergence of psychopathological computations in large language models". Codes with S3AE architecture and u…
Access
Public
Updated Apr 14, 2025
Repo size
679 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt679 MB · 100%
At a glance
- License
- cc-by-nc-4.0
- Access
- Public
- Created
- Apr 14, 2025
- Updated
- Apr 14, 2025
- SHA
- fbe8b365
Model card
From the Hugging Face model README
This is trained parameters of the Sentence-level, Supervised, Sparse AutoEncoder (S3AE) proposed in the paper "Emergence of psychopathological computations in large language models". Codes with S3AE architecture and use examples can be found in this Github.
S3AE was trained on the residual stream in the 10th layer of instruction-tuned Gemma 2 27B, using a proprietary synthetic dataset with psychopathology symptom labels. The model weight precision is bfloat16, and the hidden dimension size is 8 times that of the LLM residual stream.
The 1st to 17th dimensions of S3AE hidden features, respectively, correspond to activations of the following thoughts:
1: 'depressed mood',
2: 'anhedonia (loss of interest)',
3: 'pessimism',
4: 'guilt',
5: 'anxiety',
6: 'catastrophic thinking',
7: 'perfectionism',
8: 'active avoidance',
9: 'grandiosity (delusion of grandeur)',
10: 'manic mood',
11: 'impulsivity',
12: 'risk-seeking',
13: 'splitting (binary thinking)',
14: 'unstable self-image',
15: 'aggression',
16: 'anger',
17: 'irritability'.
Dimensions 7, 13, and 14 were not used for the paper's analysis.
- License
- cc-by-nc-4.0
- Languages
- en
- Created
- Apr 14, 2025
- Updated
- Apr 14, 2025