Task000608_Upright_ViTacFormer_H100_B512_Hand_Intern

Task000608 Upright Water Bottle — SH5 ViTacFormer H100, B512

This public package contains two inference-only checkpoints from the same scratch run:

• checkpoints/best_validation.pt: step 100,000, selected by the lowest recorded composite validation score.
• checkpoints/latest_model.pt: step 144,784, exact model weights extracted from the resumable latest checkpoint; optimizer and RNG are intentionally omitted.

Default loader selection is best_validation.pt. The latest checkpoint is included for comparison, not because it is safer or better. This is an incomplete research checkpoint and is not approved for powered SH5 deployment.

Selection evidence

Training stopped before the planned 300k predicted-tactile phase. Right tactile did not beat persistence and the combined offline release gate failed. Training loss is not deployment evidence. Review validation_history.json, train_config.json, and artifact_receipt.json before selecting a checkpoint.

Load and run non-actuating inference

Keep the complete snapshot together. This is a custom PyTorch loader, not a Transformers AutoModel or a standard LeRobot policy.

import sys
from pathlib import Path
from huggingface_hub import snapshot_download

root = Path(snapshot_download(repo_id="Dongkkka/Task000608_Upright_ViTacFormer_H100_B512_Hand_Intern"))
sys.path[:0] = [str(root), str(root / "source_snapshot/ViTacFormer_SH5")]
from inference_loader import load_run, predict_normalized
from preprocess import prepare_observation

# Default: validation-best. For latest, pass checkpoint="latest_model.pt".
policy, stats, config = load_run(root, checkpoint="best_validation.pt", device="cuda")
batch = prepare_observation(
    image_rgb, state_history, tactile_raw_history, tactile_baseline, stats, device="cuda"
)
actions_rad, future_tactile_normalized = predict_normalized(policy, stats, batch)
# actions_rad: [1,100,54], absolute radians
# future_tactile_normalized: [1,18,180]

Required synchronized inputs and order:

• RGB uint8 [376,672,3], configured left-head camera, no flip or rotation.
• State float32 [6,54], offsets [-15,-12,-9,-6,-3,0] at 30 Hz.
• Raw tactile float32 [18,90], left 45 then right 45, offsets -17..0.
• Per-taxel tactile baseline float32 [90], calibrated with the training convention.
• Output action order: left arm 7, right arm 7, left hand 20, right hand 20.

Verify the package and run a non-actuating synthetic smoke test:

sha256sum -c SHA256SUMS
python verify_package.py

Safety and limitations

The loader applies the saved normalization, predicted-future-tactile inference path, zero latent prior, bounded joint decoder, and arm warm-start ramp. It does not implement ROS publishing, camera/state synchronization, temporal scheduling, freshness checks, whole-chunk rejection, watchdogs, E-stop logic, or robot-specific topic mapping.

Before powered deployment, verify the exact SH5 revision, joint and tactile ordering, calibration, camera mapping, control rate, inference latency, command continuity, tracking error and stopping behavior. Complete offline replay, shadow mode and guarded low-speed trials with an operator and working E-stop. offline_release_pass=false for the exported validation-best checkpoint.

Upstream/adapted source is included under Apache-2.0. Dataset or checkpoint licensing is not inferred beyond the included source license.