static-splade-trained-pruned

SPLADEX — best_proxy checkpoint

This is the best_proxy checkpoint from an inference-free SPLADE-v3-doc layer-pruned training run pruned from Cdn13/splade-multi-static-doc.

Model description

• Architecture: SPLADE-v3-doc (MLM head, log1p(ReLU(logits)).max() document pooling)
• Query encoder: Inference-free — static per-token weights (static_query_weights.pt) initialized from IDF and learned during training.
• Document encoder: Full SPLADE document side (pruned transformer layers).
• Selection criterion: Best window-average selection proxy during training (top-1 ranking score − sparsity budget penalty).

Files

Usage

import torch
from transformers import AutoTokenizer, AutoModelForMaskedLM

repo = "Cdn13/static-splade-trained-pruned"
tokenizer = AutoTokenizer.from_pretrained(repo)
model     = AutoModelForMaskedLM.from_pretrained(repo)

# Load static query weights
sqw = torch.load("static_query_weights.pt", map_location="cpu")
query_weights = sqw["query_weights"]  # shape: [vocab_size]

Note: The query representation is presence(token) * query_weights[token], computed without any forward pass through the model.