repro_dyme_20260720

DyME ChartQA Full Visual Supervision Reproduction

This is a cleaned reproduction workspace for the official DyME project, focused on the ChartQA Table 2 style Full DyME / Full Visual Supervision experiment with LLaVA-OV-S.

Scope

• Official DyME source: code/DyME
• Official commit: bf4ca908c290030526be41344b3f0ec707ef9bb0
• Base model: LLaVA-OV-S, llava-onevision-qwen2-0.5b-ov-hf
• Task: ChartQA train/test
• Training data: runtime/chartqa/train_new_prerefine.json
• Online visual supervision: enabled through RewardCalculator and ContextRefiner
• Helper LLM: Qwen/Qwen2.5-14B-Instruct-AWQ, served by an OpenAI-compatible lmdeploy API
• Same-machine layout: one 8-GPU node, GPU 7 for helper, GPUs 0-6 for DyME training/evaluation

This package includes small runtime safety fixes for online helper failures and a grpo route alias; see metadata/debug_20260724.md.

The official config uses 8 training processes and 10 epochs. On a single 8-card machine where the helper must run locally, this package defaults to 7 training processes plus one helper GPU, while preserving the official DyME hyperparameters and online visual supervision path.

Main Commands

Check the environment:

./scripts/chartqa_full_vs/check_env.sh

Start helper, watcher, and 7-GPU training:

mkdir -p runs/chartqa_full_vs_online_7gpu/logs
nohup ./run_chartqa_full_vs_7gpu_with_helper.sh \
  > runs/chartqa_full_vs_online_7gpu/logs/run_all.nohup 2>&1 &

Start only training when the helper is already healthy:

./run_chartqa_full_vs_7gpu.sh

Evaluate a checkpoint manually:

./scripts/chartqa_full_vs/eval_chartqa_checkpoint.sh \
  runs/chartqa_full_vs_online_7gpu/checkpoints/final_checkpoint

Full setup notes are in docs/CHARTQA_FULL_VS_7GPU.md.

Smoke-test the package:

/home/deepseek_VG/.conda/envs/dyme/bin/python -m pytest tests -q
./scripts/chartqa_full_vs/run_smoke_matrix.sh

For a tiny one-step model-training smoke, add RUN_TRAIN_SMOKE=1. This uses a fake OpenAI-compatible helper and is only intended to validate code paths.

What Is Not Included

Large local-only artifacts are intentionally excluded from upload and cleanup targets:

• conda/env directories
• interrupted or non-final run checkpoints
• wandb/offline logs
• SLAKE/BiomedGPT experiments
• helper logs and pid files
• model weights
• ChartQA image files

If this repo is moved to a new machine, update DYME_PRETRAINED_MODEL_PATH, HF_HOME, and the image paths inside runtime/chartqa/train_new_prerefine.json or regenerate the preprocessed ChartQA JSON.