gemma3-4b-it-plt-activations

Gemma 3 4B-IT PLT activation statistics

Cumulative top-activation statistics for all 34 language-model PLT layers of google/gemma-3-4b-it. The PLTs are TopK-48, expansion factor 64, and were post-trained for 57 million tokens (run fingerprint 29b8c94c23cd846e).

The activation collection contains 257,507 mixed training inputs and 121,021,288 processor tokens. It is cumulative but not a claim that the source datasets were exhausted.

Layout

• stats/layer_XX.safetensors: packed per-feature activation statistics.
• inputs/part-XXXXX.parquet: normalized input/token records referenced by the activation tensors.
• manifest.json: shapes, provenance, file hashes, and join metadata.

Each layer contains feature_ids, slot_counts, top_values, input_ids, token_positions, and firing_counts. top_values are BF16; IDs, positions, and slot counts are INT32; firing counts are INT64. Packed example arrays are ordered by feature, with boundaries given by the cumulative sum of slot_counts.

input_ids join to the input_id column in the Parquet files. Those records preserve the exact processed token IDs, token masks, optional multimodal token types, formatted text, example identity, and source metadata.

No image bytes are included. image_references contains stable pointers to the source images, so consumers can resolve images using their own dataset access.

Loading a layer

from huggingface_hub import hf_hub_download
from safetensors import safe_open

path = hf_hub_download(
    "Jingcheng/gemma3-4b-it-plt-activations",
    "stats/layer_12.safetensors",
)
with safe_open(path, framework="pt", device="cpu") as f:
    feature_ids = f.get_tensor("feature_ids")
    slot_counts = f.get_tensor("slot_counts")
    top_values = f.get_tensor("top_values")
    input_ids = f.get_tensor("input_ids")
    token_positions = f.get_tensor("token_positions")
    firing_counts = f.get_tensor("firing_counts")

See manifest.json for the exact model revision, source-checkpoint SHA-256, per-file hashes, feature counts, and retained-example counts.