colar-qwen25-7b-ncp-post-rl

agurung/colar-qwen25-7b-ncp-post-rl

colar-qwen25-7b-ncp-post-rl is a reinforcement learning model from agurung. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.

This repository contains a historical Qwen2.5-7B CoLAR checkpoint exported to a Hugging Face-compatible layout. The repo root works for standard decoding (Transformers/vLLM), and extrastate.pt preserves the CoLAR late…

Downloads · 30 days

18% of all-time downloads

All-time downloads

Public

Parameters

7.6B

16.5 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors15.2 GB · 92%

At a glance

Task: Reinforcement Learning
License: apache-2.0
Model type: qwen2
Access: Public
Created: Apr 9, 2026
Updated: Apr 9, 2026
SHA: 383a2b2c

Base models

Qwen/Qwen2.5-7B-Instruct

Model card

From the Hugging Face model README

CoLAR Qwen2.5-7B NCP Post-RL

This repository contains a historical Qwen2.5-7B CoLAR checkpoint exported to a Hugging Face-compatible layout. The repo root works for standard decoding (Transformers/vLLM), and extra_state.pt preserves the CoLAR latent head for latent decoding.

What Is In This Repo

Standard HF model files at repo root (config.json, model*.safetensors, tokenizer files)
extra_state.pt with the latent policy and embedding weights required for CoLAR latent decoding
export_meta.json from the local export step
latent_metadata.json with structured provenance and hyperparameters
artifacts/source_wandb_config.yaml, artifacts/source_wandb_summary.json, and artifacts/source_wandb_metadata.json
artifacts/source_all_config.json parsed from the original WandB all_config

Source Provenance

Field	Value
WandB run	`5jh51trt`
Run date	`2025-10-16`
Task	Narrative continuation planning
Stage	post-RL
Base model	`Qwen/Qwen2.5-7B-Instruct`
Git commit	`6c32ea1f905f8b3de215f9404454647f585d43d7`
Host / GPU	`alexgurung-math-4z956-cxt8m` / `NVIDIA H200`
Original checkpoint	`/mnt/volume3/baseline_colar/colar/oldlogs/colar/colar-experiments/5jh51trt/checkpoints/pt_checkpoint_best`
Initialized from	`/mnt/disk/baseline_colar/colar/logs/colar/colar-experiments/b7fyxsug/checkpoints/epoch1__step451__monitor3.139.ckpt/`
Generalization slug	`colar_ncp_post_rl`

Usage

Standard decoding

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_dir = "agurung/colar-qwen25-7b-ncp-post-rl"
model = AutoModelForCausalLM.from_pretrained(repo_dir, torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(repo_dir)

Latent decoding

from huggingface_hub import snapshot_download
from pathlib import Path

local_dir = Path(snapshot_download("agurung/colar-qwen25-7b-ncp-post-rl"))
llm_dir = local_dir
extra_state = local_dir / "extra_state.pt"

Example evaluator invocation:

python -m litereason.experiments.latent_eval.colar.evaluate_colar   --llm-dir "$LOCAL_DIR"   --extra-state "$LOCAL_DIR/extra_state.pt"   --test-file litereason/experiments/generalization/data/gsm8k.jsonl   --prompt-variant standard   --save-preds preds_gsm8k_standard.jsonl   --num-samples 5   --use-chat-template

Key Hyperparameters

Field	Value
Trainer strategy	`deepspeed_stage_2_offload`
Max epochs	`20`
Precision	`bf16`
Global batch size	`1`
Validation batch size	`4`
Compression factor	`5`
Max latent forward	`64`
Latent temperature	`1.0`
Group size	`8`
Experience batch size	`1`
Train samples / epoch	`64`

Reported Metrics

Metric	Value
`monitor`	`-1.2438700199127197`
`val/reward`	`-1.2438700199127197`
`val/acc`	`0`
`val/loss`	`14.8023681640625`
`val/ce_loss`	`12.930797576904297`
`val/embed_modeling_loss`	`1.871570110321045`
`val/output_length`	`220.71340942382812`
`val/n_latent_forward`	`5.304878234863281`
`epoch`	`19`
`trainer/global_step`	`319`

Notes

Initialized from the NCP post-SFT checkpoint b7fyxsug.
The historical RL run used DeepSpeed stage-2 offload and answer loss disabled.

Local Reference Paths

WandB config: /mnt/volume3/baseline_colar/colar/oldlogs/colar/wandb/run-20251016_113852-5jh51trt/files/config.yaml
WandB summary: /mnt/volume3/baseline_colar/colar/oldlogs/colar/wandb/run-20251016_113852-5jh51trt/files/wandb-summary.json
WandB metadata: /mnt/volume3/baseline_colar/colar/oldlogs/colar/wandb/run-20251016_113852-5jh51trt/files/wandb-metadata.json
Local export dir: /mnt/disk/baseline_colar/colar/exported_hf/5jh51trt

Task: Reinforcement Learning
Type: qwen2
License: apache-2.0
Created: Apr 9, 2026
Updated: Apr 9, 2026