tachand-v-foundation-v2-4a

TacHand V Foundation v2.4-A — Model Card

Model ID (planned): ppark-ai/tachand-v-foundation-v2-4a License: Apache 2.0 (weights) + CC-BY 4.0 / research-only (eval data references; see corpus table) Pretrain date: 2026-05-16 Maintainer: ppark-ai

---

TL;DR

Frozen-V + adapter contact-aware vision foundation. v2.4-A is the v2 release winner: frozen DINOv2-L (initialised from v2.3) + trainable T branches / v_proj / sensor token + NEW corpora (TAG, Sparsh, TacQuad) at small cap, lifts to 82.3%.

Base: DINOv2-L (Meta, 300M params), frozen Initialisation: v2.3 release-candidate checkpoint Trainable: v_proj (1024→512), T branches, sensor-type embedding Pretrained on: 153,562 paired V+T frames across 9 corpora (v1 mix replay + TAG/Sparsh/TacQuad capped) Open-loop FSL accuracy: 82.3% k=10 nn=1 (5 seeds, 10-class VTDexManip n=200, matched v1 audit protocol), +4.4pp vs v1, +2.8pp vs v2.3. Latent dim: 512 (projected) / 1024 (raw DINOv2 output) Input: 112×112 RGB

This is Layer 2 of the Open Tactile Stack. v2.4-A supersedes v2.3 as the recommended Layer 2 checkpoint for downstream ContactAdapter / VLA wrappers.

Why frozen-V + adapter. v2.1 (full NEW-corpora joint mix) and v2.2 (capped NEW-corpora joint mix) regressed VTDex by 27–38 pp under joint V-trunk training. Codex agent diagnosis: heterogeneous NEW corpora dilute the V representation through shared-trunk capacity contention. v2.4-A freezes the V trunk (preserving the v2.3 representation) and routes NEW-corpus signal through the trainable T branches, v_proj, and sensor token only. Empirically confirmed: 82.3 % vs v2.1 catastrophic 50 %. See docs/validation/v2_new_corpora_poison_failure_audit.md.

---

Architecture

RGB ──► DINOv2-L (FROZEN, v2.3-init) ──► v_proj (1024→512, TRAIN) ──► z_v
                                                                       │
Tactile (per-sensor) ──► T branch (TRAIN) ──► z_t ─────────────────────┤
Sensor-type ──────────► sensor token (TRAIN) ──► z_s ──────────────────┤
                                                                       │
                                                  InfoNCE(z_v, z_t, z_s)

The V trunk is bit-for-bit identical to v2.3; only the projection, modality branches, and sensor token receive gradient. This is the same decoupling pattern as Layer 3 ContactAdapter (a_final = a_base + gate * clip(delta)) — TacHand's signature design carried into Layer 2 pretrain.

---

Usage (3 lines)

from tachand_tools.foundation import VFoundation

v = VFoundation.load("ppark-ai/tachand-v-foundation-v2-4a")
embed = v.encode(rgb_image)                  # → torch.Tensor (512,)
embeds = v.encode_batch([img1, img2, img3])  # → (3, 512)

Frozen by default. No grad. Use as feature extractor for downstream tasks.

---

Pretrain details

Training data (153,562 paired V+T frames across 9 corpora)

Total: 153,562 frames. NEW corpora are capped so they cannot dominate the round-robin batch schedule — combined with the frozen V trunk, this is the architectural fix for the v2.1 / v2.2 poison.

Training command

python scripts/v2/pretrain_v_foundation_v2.py \
    --vtdex-episodes 1500 \
    --sim-episodes 1000 \
    --ftm-root data/freetacman_features_d256_v2 \
    --tvl-ssvtp-root data/tvl_full/tvl_dataset/ssvtp \
    --tvl-hct-root data/tvl_full/tvl_dataset/hct \
    --ofreal-root data/objectfolder_real/extracted --ofreal-max-obj 50 \
    --tag-root data/touch_and_go --tag-max-frames 833 \
    --sparsh-root data/sparsh/touch_slide --sparsh-max-pkls 64 \
    --tacquad-root data/tacquad --tacquad-max-frames 1829 \
    --epochs 4 --batch-size 64 --lr 1e-4 \
    --embed-dim 512 --vision dinov2-l --image-size 112 \
    --sensor-token \
    --freeze-vision \
    --init-ckpt results/v_foundation_v2.3/checkpoint.pt \
    --output results/v_foundation_v2.4a_frozen

Flags landed in commit cdb24f4 (--freeze-vision, --freeze-v-proj, --init-ckpt).

Reproducibility

Per-class audit (3 seeds, 10-class mean): 80.8 ± 5.3 %.

Checkpoint: /home/patrick/results/v_foundation_v2.4a_frozen/checkpoint.pt (1.83 GB) Checkpoint SHA: <v2.4-A ckpt commit pending>

---

Evaluation

Protocol: 10-class VTDexManip open-loop FSL, k-NN retrieval, 5 seeds × n=200, matched v1 audit protocol.

Per-class table (3 seeds, n=200, mean ± std)

Worst-class lift (Scissors / Pressing) is the headline robustness signal: v1's two structural weaknesses both move up.

Cross-corpus held-out (held_out_30pct vs raw DINOv2-L)

Cross-corpus is not yet positive for TAG / TacQuad. v2.4-A is an in-domain win with cross-corpus partial recovery vs v2.1 / v2.2 (which were larger negatives); the +5 pp cross-corpus gate remains open for v2.5.

---

Intended use

1. Frozen feature extractor for V/T downstream tasks (FSL, classification, retrieval).
2. Layer 2 of Open Tactile Stack: input to ContactAdapter + robot company VLA.
3. Recommended Layer 2 checkpoint: v2.4-A supersedes v1 / v2.3 for new downstream work.

Out-of-scope

• Cross-corpus claims on TAG / TacQuad — Δ is still negative vs raw DINOv2-L on those held-outs; use raw DINOv2-L if your evaluation is TAG- or TacQuad-only.
• Direct fine-tuning of the V trunk (it's frozen by design; the architectural reason is the v2.1 / v2.2 poison).
• Closed-loop policy on its own (use ContactAwareWrapper + ContactAdapter).
• Sensor-specific calibration (use SensorAdapter for sensor → standard latent).

Limitations

• Cross-corpus partial recovery. TAG −3.7 pp, TacQuad −1.5 pp vs raw DINOv2-L. v2.5 next steps: per-corpus adapter heads, larger NEW-corpus cap with stronger frozen-V isolation, or sensor-token re-balancing.
• NeuralFeels +0 pp is trivial: truly-unseen RGB held-out, both arms at chance under the current protocol; not a v2.4-A signal either way.
• Open-loop FSL ≠ closed-loop success (memory: MSE↛success). Use governance gating.
• Headline is on a 10-class VTDexManip subset; broader-class evaluation pending.
• n = 5 seeds is the floor for v2.4-A; per-class table is n = 3.
• Wall time, training data, and ckpt size were measured on the dev GPU (RTX 5090); HF Hub publish + checkpoint SHA fill follow once private staging clears.

---

v2 ablation history

Full audit: docs/validation/v2_new_corpora_poison_failure_audit.md. Process post-mortem: docs/release/v_foundation_v2_4a_failure_to_success_narrative.md.

---

Citation

@misc{park2026tachand_v2_4a,
  author       = {Park, Soohwan and Patrick},
  title        = {{TacHand V Foundation v2.4-A}: Frozen DINOv2-L + Adapter
                  for Heterogeneous Tactile Corpora},
  year         = {2026},
  month        = may,
  howpublished = {\url{https://huggingface.co/ppark-ai/tachand-v-foundation-v2-4a}},
  note         = {Open Tactile Stack, Layer 2}
}

Built on top of v2.3 (which is built on v1); see docs/release/v_foundation_v1_card.md and docs/release/v_foundation_v2_3_model_card.md for full ancestry and underlying DINOv2 / VTDexManip / TVL / ObjectFolder / Touch-and-Go / Sparsh / TacQuad citations.

---

Layer 3 ContactAdapter (wrapper evidence)

Frozen v2.4-A V Foundation + ContactResidualHead (LayerNorm + 2-layer MLP, ~165 K trainable params) + ActionSpec L2 gate. Strong-eval: 8 DAgger iters × 30 rollouts/iter × 3 tasks × single seed = 720 paired wrapped-vs-baseline rollouts.

Architecture: frozen v2.4-A trunk → z_v (512) + proprio (16) + phase (1) → LayerNorm → MLP (192 hidden, SiLU, 2 layers) → action chunk K=8 × action_dim=16. Trainable parameters ~165 K.

ActionSpec gate: L2_EE_GRIPPER_RESIDUAL with max_residual_norm = 0.20 enforced. can_inject_residual(action_spec) is the hard gate — unknown action semantics fall back to a_base. Per inference: residual = head_out[0] - a_base; a_final = a_base + clip(residual, ±0.05).

Caveats (strong-eval-updated):

1. Statistical sample size. Single seed, 720 paired rollouts (30 × 8 × 3); consistent over 5 post-warmup iters for carry_full_cup (every iter > 0) and pick_and_place (4 iters at 100 %), not single-iter peak. Multi-seed replication is the next promotion gate.
2. wipe_table 0 % / 0 % is the documented sim physics limit (oracle-only task); baseline also fails — wrapper does not regress.
3. pick_and_place baseline 88.6 % mean leaves ≤11 pp absolute headroom; the +9.4 pp lift is robust over that bounded range.
4. a_base = cur_proprio is a sim placeholder. Deployment uses a real VLA's action as a_base; the residual composition is the same shape but transfer is unverified.
5. Init pretrain uses zero-RGB embedding (sim H5 demos predate V Foundation, do not store RGB); 8 DAgger iters with real per-step RGB mitigate this — the carry_full_cup lift appears consistently across iters 4-8, well after the first relabel pass.

Full method, training command, and per-iter table: docs/release/v_foundation_v2_4a_contact_adapter_evidence.md. Strong-eval source: ~/results/contact_adapter_v2_4a_strong/train_summary.json and ~/contact_adapter_v2_4a_strong.log (GPU server).