Downloads · 30 days
10
16% of all-time downloads
mambarl-m3-grpo-1024
Lixing-Li/mambarl-m3-grpo-1024
mambarl-m3-grpo-1024 is a machine learning model from Lixing-Li. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
A checkpoint from MambaRL: does outcome-only RL extend the effective context-retrieval horizon of a pure Mamba-2 LM, without changing the architecture? No attention is added and the recurrent state is not grown — only…
All-time downloads
63
Public
Parameters
2.7B
5.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.4 GB · 100%
At a glance
- Library
- transformers
- Access
- Public
- Created
- Aug 28, 2026
- Updated
- Aug 28, 2026
- SHA
- 5c5f9c75
Base models
Model card
From the Hugging Face model README
mambarl-m3-grpo-1024
A checkpoint from MambaRL: does outcome-only RL extend the effective context-retrieval horizon of a pure Mamba-2 LM, without changing the architecture? No attention is added and the recurrent state is not grown — only what the policy learns to keep in it changes.
- Base model:
Lixing-Li/mambarl-m2-grpo-512 - LoRA adapter merged in from
checkpoints/M3_grpo_1024(targets:in_projonly — PEFT refusesout_proj/conv1don mamba2 because the fused kernels read those weights directly) - Weights here are merged, so
AutoModelForCausalLM.from_pretrainedis enough.
Phase 2 (proposal 2.8), stage 1 at 1024 tokens: 512 GRPO steps, did not reach the promotion threshold (dev_acc=0.510, format_acc=0.980).
Answers follow a <think>...</think><answer>...</answer> contract.
- Library
- transformers
- Created
- Aug 28, 2026
- Updated
- Aug 28, 2026