Downloads · 30 days
15
22% of all-time downloads
octopusmegalopod/cram-2.6b-cjke-reasoning-sft
cram-2.6b-cjke-reasoning-sft is a machine learning model from octopusmegalopod. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
CRAM is a tiny experimental model for CJKE literary reasoning. It was trained using LFM2.6B as the pretrained baseline. Note this is only an intermediate checkpoint; advanced RL comes later (still shaping the rewards…
Downloads · 30 days
15
22% of all-time downloads
All-time downloads
67
Public
Parameters
2.6B
5.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors5.1 GB · 100%
From the Hugging Face model README
CRAM is a tiny experimental model for CJKE literary reasoning. It was trained using LFM2.6B as the pretrained baseline. Note this is only an intermediate checkpoint; advanced RL comes later (still shaping the rewards - likely comparative judges unless someone has a better idea.)
There is growing research implying that training for logic can have a negative impact on creativity. LFM2.6 was chosen over LFM2.6-Exp as Exp seems to be more inclined towards logical tasks (e.g: math) whereas the focus here is on creative writing.
CRAM was made primarily for creative / literary Chinese/Japanese/Korean to English tasks, such as translation. But it can output in those other languages too if so desired, or at least as well as can be expected for a tiny LLM.
⚠️ Note that CRAM has zero safety filters - it will write without censorship or safety alignment in mind. I may experiment with safety alignment in the RL phase later on.
CRAM also supports several different reasoning modes. In either mode, a thinking trace will be initially generated with <|cot_start|>...<|cot_end|>
Reasoning is typically associated with logical tasks (such as math), but I think it has potential in creative writing (even though studies show otherwise...) I believe reasoning on creative writing can be seen as an extension of "memory" tokens. We sacrifice test time efficiency to give the model a more concrete "memory" of its latent representations, which may be essential when working with such a small model.
(Expect degradation if using reasoning on multi-turn conversations. Will focus on this during later stages, if I get around to it.)
For high reasoning, prepend the first user message (system messages are not supported) with the text <|reserved_488|>
For low reasoning (inspired by the Chain of Draft paper), prepend the first user message with <|reserved_488|><|reserved_487|>
Why these tokens? They were already reserved in LFM2.6B's vocabulary, so we can reuse them off the bat. In addition, I suspect that there was already some pretrained thinking knowledge in the <|cot|> tokens, as the LiquidAI team were experimenting with some sort of hybrid thinking mode.
For no reasoning, don't prepend either of those two reserved tokens.
Never bother with system messages. The first user message can be seen as a pseudo-system message, along with the initial request.
I want to believe some of the reasoning knowledge learned during training still "transfers" over for standard instruct tasks, but this must be tested thoroughly. Science often makes me sad.
This model was trained and was fairly receptive to Muon. Despite LFM2.6B being pretrained assumedly with AdamW, which typically gives worse results when transferring over to Muon. I chose Muon as someone told me that it can generalize better to multi-task training (less interference) and I was curious.
Following the "LoRA without Regret" post, there does indeed seem to be some merit in it. Full finetuning had similar results to properly scaled rs-LoRA.
Attempting to slice off some full attention layers had catastrophic results. At such a small scale, we need as much attention as we can get.
Infinite repetitions are still an annoying issue with a model at this scale, but I plan to address this during RL with a few experimental length penalties. A naive ngram repeition penalty "reward" is a very bad idea, as ngram repetition penalizers are well known to impact creative writing - a core task here. The repetition errors may be related to LFM's short conv attention architecture - it has some bias towards local or "nearby" positions.
Do not expect this model to do any coding, or work well in any other languages apart from CJKE. I also went out of my way to filter out any sample that had even a whiff of coding or math in them.
Data creation is a hassle like always. No public benchmarks because they're too easy to game, it's best to benchmark on your own data. On my private benchmarks it performed competitively, but don't take my word for it. Since this is creative-writing focused, following the Shisa paper, I think the best way to benchmark this is to use a Judge model and compare writings. COMET and other standard metrics does very poorly for LLMs these days.
ServiceNow I believe published a recent paper finding that we can already decipher what tool or general trajectory a reasoning LLM will go for by its latent state - even before it actually writes out the thinking tokens?! This implies that models may already "know" its reasoning, and is just writing it down afterwards. It would be interesting to see if CRAM also suffers from this quirk.