Downloads · 30 days
6
12% of all-time downloads
octopusmegalopod/cram-2.6b-cjke-reasoning-rl-v3
cram-2.6b-cjke-reasoning-rl-v3 is a machine learning model from octopusmegalopod. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Further RL post-training on CRAM. Interesting results.
Downloads · 30 days
6
12% of all-time downloads
All-time downloads
49
Public
Parameters
2.6B
5.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors5.1 GB · 100%
From the Hugging Face model README
Further RL post-training on CRAM. Interesting results.
Most of the RL was focused on CJK -> English translation tasks.
Interestingly enough, for certain translation reward functions, using LLM-as-a-Judge with 7-9B to 20-40B judges made little diifference, but kept the strongest judge possible to be safe.
Contrastive ranking rewards seem very effective, and also act as a form of regularization (as generations are always compared to each other, rather than some abstract score in some model's imagination.)
Language switching does still seem to be an issue, need further reward shaping experiments.