Downloads · 30 days
8
23% of all-time downloads
mambarl-m5-grpo-4096
Lixing-Li/mambarl-m5-grpo-4096
mambarl-m5-grpo-4096 is a machine learning model from Lixing-Li. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
A checkpoint from MambaRL: does outcome-only RL extend the effective context-retrieval horizon of a pure Mamba-2 LM, without changing the architecture? No attention is added and the recurrent state is not grown — only…
All-time downloads
35
Public
Parameters
2.7B
5.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.4 GB · 100%
At a glance
- Library
- transformers
- Access
- Public
- Created
- Aug 29, 2026
- Updated
- Aug 29, 2026
- SHA
- cf960657
Base models
Model card
From the Hugging Face model README
mambarl-m5-grpo-4096
A checkpoint from MambaRL: does outcome-only RL extend the effective context-retrieval horizon of a pure Mamba-2 LM, without changing the architecture? No attention is added and the recurrent state is not grown — only what the policy learns to keep in it changes.
- Base model:
Lixing-Li/mambarl-m4-grpo-2048 - LoRA adapter merged in from
checkpoints/M5_grpo_4096(targets:in_projonly — PEFT refusesout_proj/conv1don mamba2 because the fused kernels read those weights directly) - Weights here are merged, so
AutoModelForCausalLM.from_pretrainedis enough.
Phase 2 (proposal 2.8), stage 3 at 4096 tokens: 512 GRPO steps, did not reach the promotion threshold (dev_acc=0.000, format_acc=0.000).
Answers follow a <think>...</think><answer>...</answer> contract.
- Library
- transformers
- Created
- Aug 29, 2026
- Updated
- Aug 29, 2026