Semma-27B

Semma-27B

Overview

Semma-27B is a fine-tuned version of the Gemma2-27B base model, trained on the Psych-101 dataset. This model is specialized for tasks requiring insight into human cognition and decision-making, as captured in natural language transcripts from psychological experiments. It's well-suited for research, educational tools, or applications needing psychology-informed language generation.

Model Details

• Base Model: Gemma2-27B
• Fine-Tuning Dataset: Psych-101
• Parameters: 27 billion
• Language: English
• Intended Use: Generating responses grounded in human cognitive patterns, answering psychology-related queries, or simulating decision-making scenarios.

Dataset: Psych-101

Psych-101 is a dataset of natural language transcripts from human psychological experiments. It includes trial-by-trial data from 160 experiments, involving 60,092 participants who made 10,681,650 choices. Human decisions are encapsulated within << and >> tokens.

• Source Paper: Centaur: A Foundation Model of Human Cognition
• Point of Contact: Marcel Binz
• Languages: English
• Data Fields:
  • text: Natural language transcription of the experiment
  • experiment: Identifier for the experiment
  • participant: Identifier for the participant
• Access: Available via datasets.load_dataset('marcelbinz/Psych-101')

Example Prompt from Psych-101

You will be presented with triplets of objects, assigned to the keys D, P, and H.
In each trial, indicate which object is the odd one out by pressing the corresponding key.
Choose the object least similar to the other two.

D: piecrust, P: game, H: bracelet. You press <<D>>.
D: tuning fork, P: rocket, H: waffle iron. You press <<P>>.
D: grits, P: combination lock, H: suitcase. You press <<D>>.
D: boulder, P: odometer, H: salami. You press <<P>>.
D: spoon, P: diaper, H: satellite dish. You press <<P>>.
[...]

How to Use

Load Semma-27B from Hugging Face using the Transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "marro-co/semma-27b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Example inference
input_text = "Which is the odd one out: D: piecrust, P: game, H: bracelet?"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))  # Expected: <<D>>

Fine-Tuning Process

Semma-27B was fine-tuned on Psych-101 to capture patterns in human cognitive choices and their natural language context. The process emphasized preserving the base model's general capabilities while enhancing its performance on psychology-specific tasks, such as identifying odd-one-out selections.

Limitations

The model may overfit to Psych-101's experimental structure and struggle with unrelated domains. Outputs should be validated, especially for applications beyond cognitive psychology.

Acknowledgments

• Built upon Gemma2-27B by Google DeepMind
• Fine-tuned on Psych-101, curated by Marcel Binz (Centaur: A Foundation Model of Human Cognition)

Contact

For questions, reach out at marro-co on Hugging Face.