theo_qwen2.5-7b-it_impulsive-oct-repro-lora

Status: PILOT / comparison only — not a paper organism. Older 7B reproduction of OCT, unrelated to the paper's oct_behaviour v3 organisms.

OCT impulsiveness reproduction — Qwen2.5-7B-Instruct

Reproductions of Open Character Training's published impulsiveness adapter (maius/qwen-2.5-7b-it-personas, subfolder impulsiveness), built with OCT's own pinned code (maiush/OpenCharacterTraining @ d1da9f0, maiush/OpenRLHF @ eaf40e1) by scripts/oct_repro.sh in the MO_evals repo.

This repo holds several reproductions built on different dependency stacks. They are not interchangeable. Read the table before pulling a subfolder.

Which adapter to use

Published reference, measured the same way: 117.885.

Why keep/qwen2.5-7b-it is deprecated

It was trained with deepspeed==0.18.0, which openrlhf's requirements.txt pins. In the window 0.17.6 – 0.18.2, DeepSpeed ZeRO stage 1/2 without CPU offload silently discards every micro-batch but the last of each gradient-accumulation window: at runtime/zero/stage_1_and_2.py:854 the overwrite-vs-accumulate branch tests averaged_gradients while the running total had moved to all_grad_tensors, so the add_ never runs. This recipe accumulates 32 deep, so that adapter was trained at an effective batch size of one.

Introduced by DeepSpeed PR #7509 (merged 2025-08-26), fixed by PR #7665 (merged 2025-11-19). In released versions the window is 0.17.6 (2025-09-19) through 0.18.2 (2025-11-05), closed by 0.18.3 (2025-12-09) — dates from the PyPI release index. There is no error and no warning, and the loss curve looks healthy.

OCT's own published artifact is unaffected. It was uploaded 2025-09-06, when the newest DeepSpeed release was 0.17.5 (2025-08-20) — below the window. That is also why the matched-sept2025 rollback above reproduces it: pinning the September-2025 stack pins deepspeed 0.17.5, which is simply what upstream would have had.

The adapter is kept rather than deleted because it is the evidence for that finding — the 0.617× weights are what proved the bug reached us. Do not use it as a model organism.

The remaining subfolders are experiment artifacts, not organisms

stages/, lib-bisect/, stack-parity/, sft-hypothesis/, epochs-probe/, peft-bisect/ and probe/ are intermediate stages and single-variable probes from the investigation that identified the cause. They exist so the measurements can be re-derived; none is a finished organism.

Reproducing

scripts/oct_repro.sh now installs deepspeed>=0.18.3 after openrlhf (whose requirements would otherwise pull 0.18.0) and asserts at runtime that the resolved version is outside the broken window. Write-up: docs/oct.md in the MO_evals repo.