Downloads · 30 days
6
26% of all-time downloads
lxxexxbxx/gemma3-4b-ko-rlaif-reward
gemma3-4b-ko-rlaif-reward is a machine learning model from lxxexxbxx. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as gemma.
google/gemma-3-4b-it 기반 보상 모델(Reward Model) LoRA 어댑터
Downloads · 30 days
6
26% of all-time downloads
All-time downloads
23
Public
Repo size
57.3 MB
Likes
0
Public
Click a slice to open those files.
.json33.4 MB · 58%
From the Hugging Face model README
google/gemma-3-4b-it 기반 보상 모델(Reward Model) LoRA 어댑터
google/gemma-3-4b-it (4-bit 양자화)SEQ_CLS (시퀀스 분류)r=16, lora_alpha=32, lora_dropout=0.1q_proj, k_proj, v_proj, o_projscore (점수 출력 레이어를 학습 대상에 포함)RewardTrainer + PEFT분류 헤드 미구성
AutoModelForSequenceClassification 로드 시 score 헤드가 자동 구성되지 않음modules_to_save에 포함시켜 학습 대상으로 지정token_type_ids 요구
token_type_ids 필드 필요GemmaRewardCollator 작성import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import PeftModel
base = AutoModelForSequenceClassification.from_pretrained(
"google/gemma-3-4b-it",
num_labels=1,
device_map="auto",
)
model = PeftModel.from_pretrained(base, "lxxexxbxx/gemma3-4b-ko-rlaif-reward")
tokenizer = AutoTokenizer.from_pretrained("lxxexxbxx/gemma3-4b-ko-rlaif-reward")
text = "질문: ...\n답변: ..."
inputs = tokenizer(text, return_tensors="pt").to(model.device)
if "token_type_ids" not in inputs:
inputs["token_type_ids"] = torch.zeros_like(inputs["input_ids"])
with torch.no_grad():
score = model(**inputs).logits[0].item()
print(f"reward score: {score}")
google/gemma-3-4b-it의 Gemma Terms of Use 적용