Downloads · 30 days
7
33% of all-time downloads
jetson-flat-cache-cpt-adapter
mohith-das/jetson-flat-cache-cpt-adapter
jetson-flat-cache-cpt-adapter is a text generation model from mohith-das. Use it when you need the model to write or continue text. It is set up for peft.
LoRA adapter from the Continual Pre-Training (CPT) stabilization pass in Phase 2 of the llmdistillery project. Trained on top of a Qwen2.5-0.5B model whose attention layers were surgically remapped to an alternating S…
All-time downloads
21
Public
Repo size
24.6 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json11.4 MB · 46%
At a glance
- Task
- Text Generation
- Library
- peft
- Access
- Public
- Created
- Jul 25, 2026
- Updated
- Jul 25, 2026
- SHA
- 1fc05482
Try a prompt
Model card
From the Hugging Face model README
jetson-flat-cache-cpt-adapter
LoRA adapter from the Continual Pre-Training (CPT) stabilization pass in Phase 2 of the llm_distillery project. Trained on top of a Qwen2.5-0.5B model whose attention layers were surgically remapped to an alternating Sliding-Window Attention (SWA) / Gated Linear Attention (GLA) hybrid, to stabilize the newly-introduced GLA decay parameters and let the model adapt to its new attention mechanism.
- Base model (pre-surgery):
mohith-das/jetson-assistant-0.5b - 200 steps, LoRA r=8/alpha=16 on
q_proj,k_proj,v_proj,o_proj, plus the GLA layers'log_decayparameters unfrozen - Trained on Jetson Orin Nano Super (JetPack 7.2, CUDA 13.2)
See the merged full-precision model at
mohith-das/jetson-hybrid-flat-cache-0.5b for the ready-to-use checkpoint
with this adapter already folded in.
- Task
- Text Generation
- Library
- peft
- Created
- Jul 25, 2026
- Updated
- Jul 25, 2026