crow-1b-attempt1

Expirements in large-scale small-scale preference learning.

This one was a failure, it benchmarks horribly, despite responding okay to trivia questions in testing

falcon-rw-1b trained with PRO (preference ranking optimization, see https://arxiv.org/abs/2306.17492) on SuperMC and PRM800K (only stage 1) for 3 epochs, using my supertrainer2000 framework.

This is an expiremental model.

Benchmarks coming soon.

Hyperparameters:

• AdamW, weight decay of 0.01, otherwise default hyperparams
• Maximum LR of 1e-5
• Cosine schedule with a warmup of 5400 steps
• Batch size of 4 (2 real x 2 accumulated)
• Maximum of 5 epochs, early stopping (visual observation), stopped after 3
• Gradient clipping norm value of 1.0
• PRO beta of 4

Training prompt format:

### Query
[insert instruction here]

### Answer
[insert response here]

Open LLM Leaderboard Evaluation Results

Detailed results can be found here