Downloads · 30 days
8
14% of all-time downloads
llama2_7b_chat-WaRP-beavertails-kr0.01_lr5e-5
kmseong/llama2_7b_chat-WaRP-beavertails-kr0.01_lr5e-5
llama2_7b_chat-WaRP-beavertails-kr0.01_lr5e-5 is a machine learning model from kmseong. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama2.
kmseong/llama27b-chat-Safety-FT-lr5e-5 를 시작점으로, WaRP(Weight space Rotation Process) 재파라미터화 공간에서 안전 관련 계수 방향을 동결한 채 gsm8k (basis/mask: beavertails) 로 downstream fine-tuning 한 모델입니다.
All-time downloads
57
Public
Parameters
6.7B
13.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors13.5 GB · 100%
At a glance
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Jul 28, 2026
- Updated
- Jul 28, 2026
- SHA
- f702059b
Base models
Model card
From the Hugging Face model README
Safety-WaRP (WSR-Tune) — gsm8k (basis/mask: beavertails) fine-tuned keep_ratio=0.01
kmseong/llama2_7b-chat-Safety-FT-lr5e-5 를 시작점으로, WaRP(Weight space Rotation Process) 재파라미터화 공간에서
안전 관련 계수 방향을 동결한 채 gsm8k (basis/mask: beavertails) 로 downstream fine-tuning 한 모델입니다.
- 각 weight matrix 를 입력 활성값 공분산의 고유기저
U로 회전 (C = W U) - 안전 데이터(circuit_breakers)에 대한 gradient 중요도 상위
keep_ratio좌표를 동결 - 나머지("flat") 좌표만 학습 — forward 의 mask+detach 로 구현 (non-freeze 방식)
- token-wise constrained SFT (shallow-vs-deep) 결합
적용 범위: q_proj, k_proj, v_proj, up_proj, down_proj / 전체 32개 layer / per-layer 중요도.
Training run
| base model | kmseong/llama2_7b-chat-Safety-FT-lr5e-5 |
| downstream data | gsm8k (basis/mask: beavertails) (7473 samples) |
| epochs / lr | 3 / 5e-05 |
| batch x grad_accum | 2 x 8 (effective 16) |
| optimizer / scheduler | adamw_torch / cosine |
| coordinate space | non_freeze |
| frozen safety coefficients | 45,681,460 / 4,496,293,888 (1.02%) |
| train wall-clock | 1964 s |
| train peak VRAM (device) | 96.02 GB |
- Type
- llama
- License
- llama2
- Created
- Jul 28, 2026
- Updated
- Jul 28, 2026