colar-code-cruxreal-l1b

CoLaR-code — latent reasoning on code execution, trained on real public data

Latent-reasoning checkpoint for code output prediction (execution reasoning), trained on a three-tier difficulty ladder built entirely from real public datasets. No synthetic data.

Task

Given a Python function and a concrete call, predict the returned value.

Given the Python function:
def f(s):
    return len(s) == s.count('0') + s.count('1')

What is the value returned by f('102')? Answer with the exact returned value.
→ False

Chosen deliberately as a non-arithmetic reasoning domain, to test whether latent-depth findings generalise beyond math.

Training data

1488 train / 165 val, seed 0. Tier distribution: T0 562 · T1 612 · T2 479.

Difficulty ladder: depth_proxy = #lines + 8 × #loops for CRUXEval/MBPP (split at the median into T0/T1); LiveCodeBench uses its native numsteps and forms T2 wholesale. The ladder is an ordinal scale — absolute values are not compared across sources.

Scope note on the reasoning chain. Questions and answers are 100% real public data. The intermediate steps are derived mechanically from the real source code (the function's own body lines, capped at 7, plus one templated concluding sentence). They are not human- or model-written chains of thought — please describe them accurately when citing.

Recipe

Field-for-field identical to the earlier synthetic-data run's hparams.yaml — the only change is the data.

Training loss: total 6.210 → 0.169 · ce 4.960 → 0.006 · embed_modeling 1.250 → 0.162.

Verification (RTX 3060, N=90 held-out)

Honest boundary

The latent carries content (swapping in another problem's latent changes the answer 84.4% of the time) but correctness does not depend on it (ON−OFF ≈ 0). This checkpoint is suitable as a platform for latent-space intervention studies — where the criterion is that the latent carries content — but no accuracy claim should be made. At 1488 rows / ~0.25M tokens, overfitting is expected; 0.233 is not a performance result.

Loading

export TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1

COLAR_EMB_STD=0.018 (Llama-1B) · COMPRESS=5 · separator token ###.