C_SERVER-GRPO-MERGED

C_SERVER-GRPO-MERGED

This repository packages a GRPO LoRA adapter merged into CELL-LAB/lora-plus-f2f-backup.

Two folders are included:

• merged/: standalone BF16 checkpoint for vLLM serving.
• adapter/: PEFT LoRA adapter files kept for reproducibility and adapter-only loading.

The LoRA has already been merged in merged/. Do not pass an adapter or enable LoRA when loading the merged/ folder.

vLLM 0.8.1

This repository is prepared for the requested serving stack:

pip install "vllm==0.8.1" "transformers==4.50.0"

Because the full checkpoint is stored in the merged/ subfolder, download the repository first and point vllm serve at that local folder:

python - <<'PY'
from huggingface_hub import snapshot_download
snapshot_download("CELL-LAB/C_SERVER-GRPO-MERGED", local_dir="./C_SERVER-GRPO-MERGED")
PY

vllm serve ./C_SERVER-GRPO-MERGED/merged \
  --served-model-name C_SERVER-GRPO-MERGED \
  --dtype bfloat16 \
  --max-model-len 8192 \
  --trust-remote-code

For multiple GPUs, add --tensor-parallel-size GPU_COUNT.

The BF16 weight shards are large. Make sure the GPU has enough memory for the model weights plus KV cache.

Direct Python vLLM

from huggingface_hub import snapshot_download
from vllm import LLM, SamplingParams

repo_dir = snapshot_download("CELL-LAB/C_SERVER-GRPO-MERGED")
model_dir = repo_dir + "/merged"

llm = LLM(model=model_dir, trust_remote_code=True, dtype="bfloat16")
sampling = SamplingParams(max_tokens=512, temperature=0.0)
outputs = llm.generate(["안녕하세요"], sampling)
print(outputs[0].outputs[0].text)

Adapter Loading

from transformers import AutoTokenizer, Gemma3ForConditionalGeneration
from peft import PeftModel

base_id = "CELL-LAB/lora-plus-f2f-backup"
adapter_dir = "adapter"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base = Gemma3ForConditionalGeneration.from_pretrained(
    base_id,
    device_map="auto",
    torch_dtype="auto",
)
model = PeftModel.from_pretrained(base, adapter_dir)

The repository root is a container layout. For vLLM, use the merged/ subfolder, not the repo root.